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Preface 


Co-occurring  substance  abuse  and  psychological  health  disorders  are 
increasingly  common  in  military  populations.  These  co-occurring  dis¬ 
orders  (CODs)  pose  a  growing  concern  for  the  health  and  readiness  of 
U.S.  military  service  members.  Evidence  suggests  that  integrated  treat¬ 
ment  of  CODs — that  is,  treatment  that  brings  together  the  treatment 
of  both  the  substance  use  and  psychological  health  disorders — may  be 
more  effective  than  treating  each  disorder  separately  or  sequentially. 

The  Hazelden  Co-Occurring  Disorders  Program  (CODP)  is  a  cli¬ 
nician  training  program  designed  to  train  clinicians  to  provide  inte¬ 
grated  treatment  for  individuals  with  CODs.  The  CODP  will  herein 
refer  to  Hazelden’s  approach  to  treating  CODs,  which  involves  both 
CODP  training  for  clinicians  as  well  as  CODP  materials.  Recognizing 
the  increasing  need  for  effective  treatment  for  CODs  among  its  service 
members,  the  Navy’s  Bureau  of  Medicine  contracted  with  Hazelden  to 
train  staff  within  Navy  Substance  Abuse  Rehabilitation  Programs  to 
deliver  the  CODP.  The  goal  of  the  training  program  was  to  help  Navy 
clinicians  respond  to  the  needs  of  persons  who  have  both  substance 
abuse  and  psychological  health  disorders. 

To  assess  the  effectiveness  of  this  training,  RAND  researchers 
conducted  an  implementation  evaluation  of  the  program.  RAND’s 
evaluation  of  the  Navy’s  CODP  program  had  two  components:  (1)  to 
understand  the  CODP  training  approach  and  program  goals  and  (2) 
to  describe  the  implementation  of  the  CODP,  including  (a)  describing 
the  Hazelden  training  program  and  materials  and  trainee  perceptions 
of  the  training,  (b)  identifying  which  program  elements  were  imple¬ 
mented  and  sustained  at  treatment  sites,  (c)  identifying  facilitators  and 
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barriers  to  successful  implementation  of  the  CODP,  and  (d)  describing 
programs’  capability  to  provide  integrated  care. 

The  results  of  this  report  should  be  of  particular  interest  to 
national  policymakers  within  the  Department  of  Defense  and  within 
the  specific  military  services  who  are  working  to  maintain  the  readiness 
and  psychological  health  of  service  members  and  who  are  interested  in 
approaches  to  improving  care  for  service  members  with  co-occurring 
psychological  health  and  substance  abuse  disorders.  This  report  is  one 
of  a  series  of  program  evaluations  conducted  as  part  of  the  “Innova¬ 
tive  Practices  for  Psychological  Health  and  Traumatic  Brain  Injury” 
project;  for  more  information  and  to  access  other  products  from  this 
project,  please  visit  the  project  web  page  (http://www.rand.org/multi/ 
military/innovative-practices.html). 

This  research  was  sponsored  by  the  Defense  Centers  of  Excel¬ 
lence  for  Psychological  Health  and  Traumatic  Brain  Injury  (DCoE) 
and  conducted  within  the  Eorces  and  Resources  Policy  Center  of 
the  RAND  National  Defense  Research  Institute,  a  federally  funded 
research  and  development  center  sponsored  by  the  Office  of  the  Sec¬ 
retary  of  Defense,  the  Joint  Staff,  the  Unified  Combatant  Com¬ 
mands,  the  Navy,  the  Marine  Corps,  the  defense  agencies,  and  the 
defense  Intelligence  Community.  Eor  more  information  on  the  RAND 
forces  and  Resources  Policy  Center,  see  http://www.rand.org/nsrd/ 
ndri/centers/frp.html  or  contact  the  director  (contact  information  is 
provided  on  the  web  page). 
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Summary 


Background 

In  recent  years,  the  prevalence  of  psychological  health  conditions 
among  military  service  members  has  increased  substantially  (Institute 
of  Medicine,  2013).  Posttraumatic  stress  disorder  (PTSD),  for  example, 
has  emerged  as  one  of  the  most  common  psychological  health  condi¬ 
tions  among  younger  veterans  who  were  deployed  to  Iraq  and  Afghani¬ 
stan  in  Operation  Enduring  Freedom,  Operation  Iraqi  Freedom,  and 
Operation  New  Dawn.  Studies  have  shown  that  symptoms  of  PTSD 
among  veterans  frequently  co-occur  with  substance  use  problems,  lead¬ 
ing  to  poorer  overall  health,  and  a  greater  difficulty  reintegrating  into 
civilian  life. 

Patients  with  co-occurring  psychological  health  and  substance 
abuse  conditions  have  unique  treatment  needs,  and  evidence  suggests 
that  integrated  treatment  for  these  conditions — that  is,  treatment  that 
brings  together  the  treatment  of  both  the  substance  use  and  psycholog¬ 
ical  health  disorders — may  be  more  effective  than  treating  each  condi¬ 
tion  separately  or  sequentially.  Patients  who  do  not  receive  integrated 
care  for  co-occurring  disorders  (CODs)  are  at  risk  for  poorer  outcomes 
chiefly  because  the  care  provided  does  not  address  the  interrelationship 
between  the  psychological  health  and  substance  use  disorders. 

Recognizing  the  need  for  integrated  treatment  of  CODs  among 
Navy  service  members,  the  Navy’s  Bureau  of  Medicine  (BUMED) 
contracted  with  Hazelden,  a  nonprofit  organization  that  specializes  in 
training  clinical  personnel,  to  deliver  an  evidence-based  intervention 
for  treating  patients  with  CODs.  The  Navy  arranged  for  Hazelden  per- 
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sonnel  to  train  staff  from  Navy  Substance  Abuse  Rehabilitation  Pro¬ 
grams  (SARPs)  to  develop  competence  in  delivering  integrated  care  for 
CODs.  SARPs  are  located  at  Navy  installations  around  the  world  and 
typically  include  a  mix  of  staff  (e.g.,  mental  health  providers,  addic¬ 
tion  counselors),  clients  (e.g.,  combat-exposed  and  nonexposed),  and 
services  provided  (e.g.,  medication  management,  assessment  services, 
continuing  care). 

The  Hazelden  training  is  known  as  the  Co-Occurring  Disorders 
Program  (CODP).  The  CODP  will  herein  refer  to  Hazelden’s  approach 
to  treating  CODs,  which  involves  both  CODP  training  for  clinicians 
as  well  as  CODP  materials.  The  CODP  is  based  on  the  evidence-based 
Integrated  Dual  Disorder  Treatment  (IDDT)  model,  which  was  devel¬ 
oped  by  the  Dartmouth  Psychiatric  Research  Center  for  people  with 
severe  psychiatric  disorders  and  co-occurring  substance  use  disorders. 
The  CODP  is  an  adaptation  of  the  IDDT,  developed  by  Hazelden  for 
individuals  with  nonsevere  mental  health  disorders.  The  IDDT  has 
been  tested  in  randomized  controlled  trials  in  civilian  (but  not  mili¬ 
tary)  populations  (Drake  et  ak,  1997,  1998).  However,  the  CODP  as 
adapted  by  Hazelden  has  not  yet  been  tested  in  a  randomized  con¬ 
trolled  trial.  This  program  consists  of  several  components  that  integrate 
evidence-based  approaches  to  treatment  for  CODs,  including  moti¬ 
vational  enhancement  therapy  (MET),  twelve-step  facilitation  (TSF), 
and  cognitive  behavioral  therapy  (CBT).  In  addition,  it  includes  mod¬ 
ules  to  guide  clinic  administrators  in  enhancing  their  clinic’s  capabil¬ 
ity  to  effectively  assess  and  treat  CODs,  to  educate  providers  about 
medication  management,  to  implement  evidence-based  assessment  and 
screening  procedures,  and  to  integrate  family  therapy  into  treatment 
when  appropriate. 

The  trainings  first  began  in  2008.  Since  then,  there  has  been  no 
systematic  evaluation  of  whether  and  how  the  CODP  has  been  imple¬ 
mented  and  the  degree  to  which  it  has  translated  into  improved  care  for 
Navy  personnel  with  CODs. 
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Purpose  of  This  Report 

RAND  researchers  evaluated  the  implementation  of  the  training 
among  SARP  personnel.  The  aims  of  our  evaluation  were  (1)  to  under¬ 
stand  the  CODP  approach  and  program  goals  and  (2)  to  describe  the 
implementation  of  the  CODP,  including  (a)  describing  the  Hazelden 
training  program  and  materials  and  trainee  perceptions  of  the  train¬ 
ing,  (b)  identifying  which  program  elements  were  implemented  and 
sustained  at  treatment  sites,  (c)  identifying  facilitators  and  barriers  to 
successful  implementation  of  the  CODP,  and  (d)  describing  programs’ 
capabilities  to  provide  integrated  care. 

This  report  presents  the  results  of  our  evaluation  and  makes  rec¬ 
ommendations  for  improving  future  training  of  SARP  personnel  to 
provide  integrated  treatment  for  CODs  as  well  as  to  improve  the  qual¬ 
ity  of  care  delivered  to  service  members  with  CODs. 


Methods 

The  evaluation  team  used  four  procedures  to  collect  data  on  the  CODP 
and  its  implementation: 

1.  Key-informant  discussions  involving  two  highly  placed  indi¬ 
viduals  with  unique  knowledge  of  the  CODP  and  the  training 
process:  the  Navy  SARP  Director  and  the  Hazelden  Manager 
of  Special  Projects,  who  served  as  Hazelden’s  main  liaison  with 
the  Navy. 

2.  A  web  survey  of  those  who  attended  the  training.  A  total  of  36 
trainees  completed  the  survey,  resulting  in  an  adjusted  response 
rate  of  32  percent  (number  of  completes  divided  by  1 14  eligible 
respondents). 

3.  Telephone  discussions  with  SARP  staff.  Seven  of  the  36  individ¬ 
uals  who  completed  the  survey  participated  in  follow-up  phone 
discussions  with  RAND  staff. 

4.  Site  visits.  RAND  staff  visited  three  SARPs,  where  we  conducted 
standardized  evaluations  of  each  site’s  capability  to  provide  inte- 
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grated  treatment  for  CODs.  Site  visits  included  observations  of 
the  program  environment,  discussions  with  SARP  staff,  dis¬ 
cussions  with  clients,  and  medical  record  reviews,  which  were 
obtained  with  informed  consent  and  without  identifying  infor¬ 
mation. 


Key  Findings 

Understanding  the  CODP  Training  Approach  and  Goals 

Key-informant  interviews  indicated  that  the  Navy’s  goal  for  the  CODP 
was  to  provide  SARP  staff  with  training  in  an  evidence-based  approach 
to  delivering  integrated  treatment  for  CODs.  The  Navy  supported  this 
approach  by  sending  SARP  staff  from  a  range  of  military  installations 
to  receive  training  at  Hazelden. 

Implementation  of  the  CODP 

Trainee  perceptions  of  the  CODP.  Trainees  generally  expressed  posi¬ 
tive  views  of  the  training.  In  particular,  informants  reported  that  the 
training  left  them  feeling  confident  about  their  ability  to  implement 
the  CODP  materials  at  SARPs.  Informants  also  generally  expressed 
the  view  that  the  training  met  or  exceeded  their  expectations.  Areas  for 
improvement  were  also  noted,  including: 

•  a  need  for  refresher  trainings  and  CODP  supervision  post-train¬ 
ing 

•  a  need  for  more  information  about  how  to  tailor  the  program  for 
specific  patient  populations,  such  as  service  members  with  high 
levels  of  combat  exposure  or  severe  mental  illness 

•  the  need  for  all  SARP  staff  to  receive  some  training  in  the  pro¬ 
gram  to  increase  service-wide  buy-in  and  support  for  the  CODP 

•  a  desire  for  more  time  observing  Hazelden  practitioners  imple¬ 
menting  the  tenets  of  the  CODP  during  training 

•  the  need  for  information  on  how  to  modify  the  approaches  taught 
by  the  CODP  for  shorter-term  treatment. 
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Program  elements  that  were  implemented  and  sustained.  The 

survey  and  interviews  indicated  that  the  most  frequently  used  program 
elements  were  the  screening  and  assessment  materials,  as  well  as  por¬ 
tions  of  the  CBT  and  ICT  (Integrating  Combined  Therapies)  curri¬ 
cula.  The  least-frequently  used  components  were  the  administrator’s 
guide  and  the  medication  management  materials.  In  general,  infor¬ 
mants  reported  that  they  generally  did  not  implement  all  elements  of 
the  approach  to  treating  COD  taught  by  CODP  but  instead  selected 
components  they  believed  would  be  most  useful  and  tailored  the  mate¬ 
rials  for  client  needs.  Some  also  expressed  the  view  that  it  would  be 
helpful  to  standardize  implementation  of  the  materials  as  much  as  pos¬ 
sible  across  SARPs. 

Facilitators  and  barriers  to  successful  implementation.  Reports 
from  all  sources  suggested  that  several  factors  aided  implementation  of 
the  COD  treatment  approach  taught  by  the  CODP.  These  included: 

•  the  high  quality  and  usability  of  the  CODP  materials 

•  sufficient  staffing,  resources,  and  time  to  implement  the  program 

•  receiving  supervision  in  how  to  implement  the  program 

•  having  SARP  counselors  with  adequate  training 

•  encouragement  from  SARP  directors,  supervisors,  and  fellow  staff 

members  to  use  the  CODP  materials. 

Barriers  reported  by  participants  included  difficulty  collaborating 
with  mental  health  clinics,  as  well  as  staff  and  leadership  resistance  to 
implementing  the  program.  In  addition,  a  low  perceived  need  for  the 
COD  treatment  approach  taught  by  the  CODP  at  certain  SARPs  led 
to  a  weaker  implementation  effort. 

The  level  of  integrated  care  delivered  at  SARPs.  Informants 
reported  that,  in  general,  most  SARPs  that  were  assessed  in  this  evalu¬ 
ation  do  not  appear  to  be  providing  integrated  care  for  CODs.  Over 
one-half  of  survey  respondents  rated  their  SARPs  as  providing  addic¬ 
tion-only  services.  Data  from  the  site  visits  were  consistent  with  this 
finding:  Of  the  three  SARPs  visited  by  RAND  researchers,  one  was 
rated  as  dual- diagnosis  capable,  while  the  other  two  were  rated  as  pro- 
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viding  addiction-only  services.  The  facilitators  and  barriers  noted  above 
were  also  related  to  SARPs’  rating  as  dual-diagnosis  capable. 


Recommendations  for  improving  Provider  Training  and 
Care  for  CODs 

Based  on  the  findings  from  all  of  our  data  gathering  efforts,  RAND 
researchers  developed  the  following  recommendations  for  improving 
the  future  training  of  clinicians  in  how  to  provide  integrated  care  for 
CODs  and  for  improving  treatment  of  service  members  with  CODs. 
While  this  evaluation  focused  on  the  CODP,  our  recommendations 
are  intended  to  apply  to  the  treatment  of  co-occurring  disorders  within 
the  military  health  system  (MHS)  more  generally.  It  was  outside  the 
scope  of  the  evaluation  to  make  recommendations  about  the  future  of 
the  CODP,  as  the  evaluation  focused  on  implementation  of  the  pro¬ 
gram  rather  than  its  effectiveness.  Rather,  these  recommendations  can 
be  used  to  inform  the  development  of  and  future  investments  in  pro¬ 
grams  like  the  CODP  that  aim  to  improve  the  delivery  of  integrated 
care  for  service  members  with  CODs. 

Improving  Provider  Training 

Recommendation  1.  Develop  a  training  plan  that  specifies  how 
the  training  will  be  implemented  and  sustained,  obtain  leadership 
support,  plan  for  staff  turnover,  provide  consultation  after  train¬ 
ing,  and  evaluate  training  efforts. 

Recommendation  la.  Select  a  clinician  training  program  that  pro¬ 
vides  ongoing  consultation  and  implementation  support.  When  selecting 
a  clinician  training,  it  is  essential  to  ensure  the  content  is  appropriate 
and  relevant  for  the  targeted  clinicians.  Training  programs  that  pro¬ 
vide  ongoing  consultation  and  implementation  support  are  more  desir¬ 
able,  as  it  is  not  unusual  for  clinicians  to  attend  a  didactic  workshop 
training  and  not  reach  competence  in  delivering  an  intervention. 

Recommendation  lb.  Obtain  organizational  support  from  leadership 
prior  to  initiating  training.  Developing  a  training  plan  in  advance  that 
includes  obtaining  “buy-in”  from  senior  leadership  (at  both  the  base 
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and  the  clinic  levels)  should  increase  the  impact  of  the  training.  SARP 
administrators  were  invited  to  attend  the  trainings,  hut  attendance  was 
optional,  so  not  all  administrators  attended. 

Recommendation  Ic.  Develop  a  consistent  approach  to  training  rel¬ 
evant  staff and  strategies  to  address  future  staff  turnover.  The  plan  should 
also  address  which  staff  should  he  trained  and  include  an  active  plan  to 
address  staff  turnover  and  reassignments.  For  the  CODP,  civilian  con¬ 
tractors  were  not  trained  because  they  could  not  travel  to  the  training. 
On-site  or  online  training  methods  would  help  to  ensure  that  contrac¬ 
tors,  who  often  stay  at  a  particular  site  for  longer  periods  than  active- 
duty  clinicians,  receive  training  as  well.  Military  staff  turnover  is  rou¬ 
tine;  therefore,  clinician  training  must  he  responsive  to  this.  Lengthy 
offsite  training,  which  tends  to  he  higher  in  cost  due  to  travel  and  time 
away,  may  not  he  the  most  appropriate  approach  for  staff  who  may  he 
reassigned  to  a  new  position  in  three  years. 

Recommendation  Id.  Provide  consultation  following  the  training. 
Likelihood  of  successful  implementation  would  he  increased  if  trainees 
received  ongoing  consultation  or  supervision  to  reach  competence. 

Recommendation  le.  Develop  an  evaluation  plan  to  support  assessing 
the  success  of  the  training.  Evaluating  the  training  is  essential  and  needs 
to  he  planned  ahead  of  the  implementation. 

Recommendation  2.  Ensure  that  clinician  training  focuses 
on  materials  and  skills  that  are  most  relevant  to  the  site’s  clinical 
practice. 

This  evaluation  suggested  that  the  training  provided  hy  Hazelden 
was  well  received.  The  CODP  training  could  he  improved  hy  focusing 
on  particular  components  of  the  materials  that  have  the  best  evidence 
base  and  are  most  relevant  to  the  site’s  clinical  practice.  As  our  findings 
showed,  some  materials  (screening  and  assessment,  CBT,  and  ICT) 
were  heavily  used,  while  others  were  used  rarely.  Given  staff  turnover 
and  challenges  in  developing  competence  in  new  clinical  skills,  a  nar¬ 
rower  focus  would  increase  the  likelihood  that  clinicians  will  be  able  to 
implement  the  new  materials  effectively. 
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Improving  Care  for  Service  Members  with  CODs 

The  ultimate  goal  of  the  training  was  to  improve  care  for  service  mem¬ 
bers  with  CODs.  The  following  recommendations  draw  from  the  evalua¬ 
tion  findings,  as  well  as  from  the  literature  on  improving  care  for  CODs. 

Recommendation  3.  Consider  requiring  that  all  service  mem¬ 
bers  who  receive  care  from  a  SARP  be  screened  for  substance  use 
and  psychological  health  problems  using  validated  measures. 

Screening  should  include  a  measure  for  alcohol  and  drug  use, 
along  with  common  co-occurring  psychological  health  conditions  such 
as  depression,  anxiety,  and  PTSD.  Discussions  with  staff  (by  phone  and 
during  site  visits)  suggested  that  SARP  staff  may  not  routinely  identify 
clients  with  CODs.  Routine  screening  of  service  members  who  enter 
formal  treatment,  and  those  who  receive  less-intensive  interventions, 
would  provide  a  mechanism  to  systematically  identify  these  clients. 

Recommendation  4.  Identify  and  certify  select  sites  as  provid¬ 
ing  enhanced  services  for  service  members  with  CODs  rather  than 
assuming  that  all  SARPs  can  provide  high-quality  care  for  CODs. 

The  evaluation  found  that  SARPs  varied  widely  in  terms  of  their 
capability  to  provide  evidence-based  care  for  CODs,  suggesting  a 
continuing  need  to  improve  quality  of  care  for  service  members  with 
CODs.  It  may  not  be  the  best  use  of  resources  to  attempt  to  train 
all  sites  to  deliver  integrated,  high-quality  care  for  CODs,  given  that 
many  sites  see  only  a  small  number  of  clients  with  these  conditions.  An 
alternative  to  attempting  to  train  all  sites  to  deliver  integrated,  high- 
quality  care  for  CODs  is  to  identify  particular  sites  that  have  the  capa¬ 
bility  to  deliver  high-quality  care  and  ensure  that  service  members  with 
CODs  get  treatment  at  those  sites.  This  may  involve  transferring  a  ser¬ 
vice  member  to  another  base  to  get  treatment,  but  we  observed  that 
this  occurs  regularly  anyway.  Individual  SARPs  could  be  reviewed  and 
certified  for  COD  care  using  a  structured  tool  like  the  Dual  Diagnosis 
Capability  in  Addiction  Treatment  (DDCAT)  assessment. 

Recommendation  5.  Implement  measures  to  assess  the  qual¬ 
ity  of  care  provided  at  SARPs,  including  both  process  and  out¬ 
come  measures. 

There  appear  to  be  few  mechanisms  in  place  for  monitoring  the 
quality  of  care  delivered  to  service  members  with  CODs,  either  at  indi- 
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vidual  SARPs  or  across  all  SARPs.  For  example,  we  did  not  identify 
any  measures  assessing  treatment  process,  which  limits  the  ability  to 
determine  whether  the  care  delivered  is  consistent  with  clinical  practice 
guidelines.  In  addition,  there  are  currently  no  systematic  approaches 
for  ongoing  monitoring  of  client  treatment  outcomes.  Outcomes  moni¬ 
toring  provides  important  information  on  the  effectiveness  of  the  ser¬ 
vices  delivered.  Further,  outcomes  monitoring  is  essential  to  delivering 
measurement-hased  care,  in  which  repeated  assessments  of  client  out¬ 
comes  help  to  guide  treatment  delivery. 


Limitations 

It  should  he  noted  that  this  evaluation  has  some  limitations  that  should 
he  considered  when  interpreting  the  results.  First,  the  implementation 
evaluation  was  initiated  approximately  two  years  after  the  last  round 
of  trainings.  While  this  post  hoc  evaluation  still  holds  value,  as  imple¬ 
mentation  of  a  new  program  takes  time  and  often  requires  organiza¬ 
tional  change  that  can  he  slow  moving,  future  evaluation  designs  may 
he  stronger  if  planned  in  advance  of  the  training  program. 

Second,  we  note  limitations  from  the  size  and  representativeness 
of  our  clinician  sample.  We  limited  the  survey  and  interview  sample 
to  trainees  with  active  duty,  reserve,  or  retired  military  status.  Due 
to  regulatory  issues,  we  were  not  able  to  incorporate  the  perspectives 
of  civilian  trainees  who  were  not  retired  military  or  spouses  of  mili¬ 
tary  personnel  (approximately  14  percent  of  trainees,  according  to  data 
obtained  from  Hazelden).  In  addition,  nearly  40  percent  of  trainees  did 
not  receive  the  invitation  to  participate  due  to  inaccurate  contact  infor¬ 
mation.  We  obtained  an  adjusted  response  rate  of  32  percent.  While 
this  response  rate  is  not  unusual  for  a  clinician  survey,  it  should  be 
noted  that  this  represents  only  16  percent  of  all  trainees.  Therefore,  the 
resulting  sample  may  be  biased  toward  clinicians  who  were  using  the 
CODP  materials  more  frequently  or  who  were  more  engaged  or  moti¬ 
vated  to  treat  CODs. 

Third,  the  data  collected  for  this  evaluation  relied  largely  on  clini¬ 
cians’  self-report  on  their  perspectives  on  the  training  and  use  of  the 
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CODP  materials.  Due  to  social  desirability  factors,  clinicians  may  be 
more  likely  to  report  favorable  attitudes  and  increased  use  of  the  CODP 
materials  that  may  not  accurately  reflect  clinical  practice.  Nonethe¬ 
less,  clinician  perspectives  still  provide  valuable  insight  on  the  utility  of 
training  and  subsequent  implementation. 

Finally,  we  note  that  we  conducted  site  visits  at  only  three  Navy 
SARPs.  Individual  SARPs  vary  in  many  ways  (e.g.,  size,  resources, 
staffing  mix)  that  could  affect  the  generalizability  of  the  site  visit 
results.  Still,  these  site  visits  provided  rich,  detailed  information  regard¬ 
ing  what  might  contribute  to  increased  implementation  of  the  CODP 
materials  and  improved  care  for  CODs.  Despite  these  limitations,  we 
believe  the  integration  of  multiple  data  sources,  both  quantitative  and 
qualitative,  provides  key  insights  into  the  potential  value  of  the  CODP 
training  program. 


Concluding  Observations 

Tfiis  report  presents  the  results  of  an  evaluation  of  a  clinician  training 
program  to  improve  care  for  Navy  personnel  with  CODs.  These  results 
suggest  that  clinicians  perceived  a  need  for  such  training,  were  open 
to  receiving  it,  and  generally  viewed  the  training  experience  positively. 
However,  the  results  also  indicate  that  more  careful  planning  and  tar¬ 
geting  of  SARPs  best  suited  to  develop  skills  for  treating  CODs  would 
improve  the  effectiveness  of  the  training  and,  therefore,  be  more  likely 
to  translate  into  higher-quality  care  for  service  members  with  CODs. 
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CHAPTER  ONE 


Introduction 


This  report  presents  the  results  of  RAND’s  evaluation  of  a  Navy  ini¬ 
tiative  to  improve  care  for  co-occurring  psychological  health  and  sub¬ 
stance  use  disorders  among  service  members. 


Co-Occurring  Disorders  Are  Increasingly  Common  in 
Military  Populations 

The  co-occurrence  of  psychological  health  and  substance  use  disorders, 
often  referred  to  as  co-occurring  disorders  (CODs),  is  common.  TTie 
estimated  prevalence  of  psychiatric  disorders  among  individuals  with 
substance  use  disorders  varies,  ranging  from  7  percent  to  35  percent 
in  general  population  studies,  and  from  17  percent  to  70  percent  in 
studies  of  clinical  samples  (Flynn  and  Brown,  2008).  Individuals  with 
COD  typically  have  worse  treatment  outcomes  and  increased  risk  of 
mortality  when  compared  with  those  with  only  one  of  these  disorders 
(Bagby,  Ryder,  and  Cristi,  2002;  Burns  and  Teesson,  2002;  Curran 
et  ah,  2007;  Watkins  et  ah,  2006).  Comorbidity  is  also  associated  with 
high  rates  of  both  attempted  and  completed  suicide  and  emergency 
department  admissions  (Aharonovich  et  ah,  2002;  Cornelius  et  ah, 
1995;  Curran  et  ah,  2003;  Lynskey  et  ah,  2004). 

Posttraumatic  stress  disorder  (PTSD),  for  example,  has  emerged 
as  one  of  the  most  common  psychological  health  conditions  among 
younger  veterans  who  were  deployed  to  Iraq  and  Afghanistan  in  Oper¬ 
ation  Enduring  Freedom,  Operation  Iraqi  Freedom,  and  Operation 
New  Dawn  (Bernhardt,  2009).  Studies  have  shown  that  symptoms  of 
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PTSD  among  veterans  frequently  co-occur  with  substance  use  prob¬ 
lems,  leading  to  poorer  overall  health  and  a  greater  difficulty  reinte¬ 
grating  into  civilian  life  (Brady  et  ah,  2009;  McDevitt-Murphy  et  ah, 
2010;  Sayer  et  ah,  2010).  When  comparing  alcohol  use  between  mili¬ 
tary  and  civilian  populations  with  PTSD,  heavy  drinking  was  found 
to  be  significantly  more  common  among  military  personnel,  especially 
among  those  who  had  been  deployed  to  combat  zones  (Bray  et  ah, 
2006).  Moreover,  significant  increases  in  alcohol  use  have  been  found 
post- deployment  compared  to  alcohol  use  pre-deployment,  indicating 
that  combat  trauma  and/or  other  deployment  experiences  may  influ¬ 
ence  the  onset  of  co-occurring  alcohol  use  disorders.  Prescription  drug 
abuse  among  military  personnel  also  appears  to  have  increased  in  the 
past  decade  (Bray  et  ah,  2010;  Servies  et  ah,  2012).  Approximately  one 
in  ten  active  duty  service  members  are  estimated  to  misuse  prescription 
drugs,  and  research  suggests  that  those  receiving  treatment  for  anxi¬ 
ety  or  depression  are  approximately  four  times  more  likely  to  misuse 
prescription  medications  compared  to  those  not  receiving  such  treat¬ 
ment  (Bray  et  ah,  2010;  Jeffery  et  ah,  2013).  Given  the  prevalence  of 
CODs  among  service  members,  a  recent  Institute  of  Medicine  report 
recommended  that  the  Department  of  Defense  (DoD)  “should  better 
integrate  care  for  SUDs  [substance  use  disorders]  with  care  for  other 
mental  health  conditions  and  ongoing  medical  care”  (2012,  pp.  9-10). 


Integrated  Treatment  for  Co-Occurring  Disorders 

Integrated  treatment  refers  broadly  to  any  mechanism  by  which  treat¬ 
ment  interventions  for  COD  are  combined  within  the  context  of  a 
primary  treatment  relationship  or  service  setting.  Integrated  treatment 
is  a  means  of  actively  combining  interventions  intended  to  address 
substance  use  and  mental  disorders  in  order  to  treat  both  disorders, 
related  problems,  and  the  whole  person  more  effectively.  Evidence  sug¬ 
gests  that  integrated  treatment  for  substance  use  and  psychological 
health  disorders  is  effective  and  may  be  superior  to  treating  the  dis¬ 
orders  separately  in  parallel  (Drake  et  ah,  2004;  Baker  et  ah,  2012; 
McCauley  et  ah,  2012;  van  Dam  et  ah,  2012).  Unfortunately,  most 
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individuals  do  not  receive  integrated  treatment  for  CODs.  Treatment 
may  be  received  for  one  disorder  only,  without  treatment  for  the  other 
disorder.  Alternatively,  treatment  is  provided  sequentially,  in  which 
one  disorder  is  treated  first,  and  then  treatment  is  initiated  for  the 
other  disorder.  Finally,  treatment  may  be  provided  in  parallel,  in  which 
both  disorders  are  treated  by  separate  providers  who  attend  to  one  dis¬ 
order  but  not  the  other,  typically  without  coordination  of  the  treatment 
of  these  disorders  (McGovern  and  McLellan,  2008).  Patients  who  do 
not  receive  integrated  care  for  CODs  are  at  risk  for  poorer  outcomes 
chiefly  because  the  care  provided  does  not  address  the  interrelationship 
between  the  psychological  health  and  SUD. 

Given  that  CODs  are  common  in  both  mental  health  and 
SUD  treatment  settings  and  that  integrated  treatment  is  most  effec¬ 
tive  for  this  population,  it  is  essential  that  treatment  settings  iden¬ 
tify  approaches  to  increasing  delivery  of  integrated  treatment.  Drake, 
Mueser,  and  Brunette  (2007)  identified  key  factors  that  are  necessary 
for  a  treatment  program  to  increase  delivery  of  integrated  treatment. 
First,  some  treatment  programs  are  designed  with  the  intent  to  provide 
treatment  solely  for  SUDs.  The  mission  and  philosophy  of  the  treat¬ 
ment  program  must  address  CODs  in  order  to  set  the  stage  for  inte¬ 
grated  treatment.  Second,  clinician  attitudes,  knowledge,  and  behavior 
must  often  change  in  order  to  transition  to  integrated  COD  treatment. 
They  emphasize  that  such  changes  require  support  and  direction  from 
program  leadership.  Third,  integrated  programs  may  need  to  be  reorga¬ 
nized.  For  example,  prescription  of  medication  may  need  to  be  handled 
by  a  single  provider  with  training  in  CODs  rather  than  by  multiple 
providers  separately  prescribing  for  each  disorder.  Further,  staff  mem¬ 
bers  need  high-quality  training  in  the  provision  of  integrated  COD 
treatment.  Finally,  Drake,  Mueser,  and  Brunette  (2007)  highlight 
quality  improvement  as  another  area  of  focus  for  programs  providing 
integrated  treatment  for  CODs.  Specifically,  it  is  recommended  that 
programs  use  repeated  assessments  to  monitor  patient  progress  to  help 
modify  treatment  for  those  not  improving.  These  key  factors  identified 
highlight  organizational  and  provider  variables  that  can  either  hinder 
or  facilitate  delivery  of  integrated  treatment  for  COD. 


4  Improving  Care  for  Co-Occurring  Psychological  and  Substance  Use  Disorders 


Department  of  the  Navy  Responses  to  Substance  Abuse 
and  Psychological  Health  Problems 

Navy  Policies 

The  need  for  improved  access  to  services  addressing  CODs  among  ser¬ 
vice  members  has  received  increasing  attention  over  the  past  decade. 
The  Institute  of  Medicine’s  2013  report  on  the  mental  health  and 
substance  abuse  needs  of  service  members  recommended  that  DoD 
improve  access  to  services  for  these  conditions  (Institute  of  Medi¬ 
cine,  2013),  and  President  Obama  recently  issued  an  executive  order 
to  improve  access  to  mental  health  services  for  all  service  members 
(Obama,  2012). 

However,  recognition  of  substance  abuse  as  a  health  condition 
that  often  comes  bundled  with  psychological  health  problems  has  only 
recently  begun  to  emerge.  Some  military  policy  views  substance  abuse 
as  an  infraction.  The  Navy’s  policy  is  no  exception.  Department  of  the 
Navy  policy  states,  “alcohol  and  drug  abuse  undermines  combat  readi¬ 
ness  and  is  incompatible  with  the  maintenance  of  high  standards  of 
performance  and  military  discipline.  It  is  a  severe  detriment  to  Navy’s 
overall  mission  readiness”  (Chief  of  Naval  Operations,  2009,  p.  2).  The 
Navy  has  a  “zero  tolerance”  illegal  drug  policy:  “Navy  members  deter¬ 
mined  to  be  using,  possessing,  promoting,  manufacturing,  or  distrib¬ 
uting  drugs  and/or  drug  abuse  paraphernalia  .  .  .  shall  be  disciplined  as 
appropriate  and  processed  for  ADSEP  [administrative  separation  from 
the  Navy]  as  required.  Members  diagnosed  as  drug  dependent  shall 
be  offered  treatment  prior  to  separation”  (Chief  of  Naval  Operations, 
2009,  p.  5).  The  median  time  from  SUD  diagnosis  to  discharge  is  133 
days  in  the  Navy,  the  shortest  of  all  of  the  services  (Servies  et  ah,  2012). 
Therefore,  service  members  with  co-occurring  drug  use  and  mental 
health  disorders  have  less  than  five  months,  on  average,  to  participate 
in  Navy-provided  treatment  before  returning  to  civilian  life.  The  same 
policy  does  not  apply  to  service  members  abusing  alcohol:  “Alcohol 
dependence  and  alcohol  abuse  are  recognized  as  treatable  conditions. 
Referral  for  treatment,  when  there  is  no  alcohol-related  misconduct .  .  . 
should  not  be  viewed  as  detrimental  when  recommending  member  for 


Introduction  5 


promotion,  command  screen,  or  special  assignment”  (Chief  of  Naval 
Operations,  2009,  p.  7). 

Navy  Substance  Abuse  and  Rehabilitation  Programs 

To  improve  access  to  treatment  for  substance  abuse,  the  Navy  has  estab¬ 
lished  Substance  Abuse  Rehabilitation  Programs  (SARPs)  at  installa¬ 
tions  around  the  world.  The  primary  focus  of  SARPs  is  the  treatment 
of  alcohol  and  drug  use  problems;  provision  of  mental  health  treatment 
at  SARPs  is  not  required  by  the  Navy.  SARPs  vary  in  number  of  staff, 
staff  mix  (e.g.,  mental  health  providers,  addiction  counselors),  client 
type  (e.g..  Marines,  combat  exposed),  number  of  clients,  and  services 
provided  (e.g.,  inpatient,  medication  management,  assessment  services, 
continuing  care).  For  example,  based  on  data  from  40  SARPs  over  the 
hrst  three  quarters  of  2012,  the  number  of  clients  screened  at  indi¬ 
vidual  SARP  clinics  ranged  from  one  to  1,268  (obtained  from  rou¬ 
tine  U.S.  Navy  Bureau  of  Medicine  and  Surgery  [BUMED]  tracking 
report).  In  the  hrst  quarter  of  2012  alone,  the  number  of  individuals 
receiving  treatment  at  each  SARP  ranged  from  zero  to  596.  SARPs  also 
have  varying  levels  of  connection  with  other  clinical  services,  including 
mental  health.  Some  SARPs  have  full-time  staff  trained  to  treat  both 
substance  use  and  mental  health  disorders,  while  others  do  not.  Fur¬ 
thermore,  SARPs  located  at  sites  with  large  medical  centers  may  have 
relatively  easy  access  to  mental  health  departments  compared  with 
those  located  within  smaller  installations.  While  we  were  unable  to 
rigorously  assess  the  collaborations  between  SARPs  and  mental  health 
departments  in  this  evaluation,  such  collaboration  may  contribute  to 
implementation  of  the  COD  treatment  approach  taught  by  the  CODP 
and  should  be  examined  in  future  research. 

Training  Naval  Medical  Staff  to  Deliver  Integrated  Care  for  CODs: 

The  Co-Occurring  Disorders  Program 

Amid  growing  recognition  of  the  need  to  integrate  care  for  service 
members  with  CODs,  BUMED  contracted  with  Hazelden,  a  nonprofit 
organization,  to  train  staff  within  SARPs  to  deliver  integrated  care  for 
CODs.  Hazelden’s  treatment  approach,  known  as  the  Co-Occurring 
Disorders  Program  (CODP)  includes  a  clinician  training  program 
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designed  to  train  clinicians  to  provide  integrated  treatment  for  indi¬ 
viduals  with  co-occurring  mental  health  and  substance  use  disorders 
(McGovern  et  ah,  2008).  “The  CODP”  will  herein  refer  to  Hazelden’s 
approach  to  treating  CODs,  which  involves  both  CODP  training 
for  clinicians  as  well  as  CODP  materials.  The  CODP  is  based  on  the 
evidence-based  Integrated  Dual  Disorder  Treatment  (IDDT)  model, 
which  was  developed  by  the  Dartmouth  Psychiatric  Research  Center 
for  people  with  severe  psychiatric  disorders  and  co-occurring  SUDs. 
The  CODP  is  an  adaptation  of  the  IDDT,  developed  by  Hazelden  for 
individuals  with  nonsevere  mental  health  disorders.  The  IDDT  has 
been  tested  in  randomized  controlled  trials  in  civilian  (but  not  mili¬ 
tary)  populations  (Drake  et  ah,  1997,  1998).  However,  the  CODP  as 
adapted  by  Hazelden  has  not  yet  been  tested  in  a  randomized  con¬ 
trolled  trial.  The  goal  of  the  CODP  is  to  “provide  an  educational  train¬ 
ing  program  that  will  enhance  the  participant’s  ability  to  respond  to 
the  needs  of  persons  who  have  both  substance  abuse  and  mental  health 
disorders”  (Hazelden  Publishing,  2008,  Goals  Tab  A:  Objectives  & 
Agenda). 


Evaluating  Navy  Implementation  of  the  CODP 

Since  the  training  program  began  in  2008,  there  has  been  no  system¬ 
atic  evaluation  of  whether  and  how  the  CODP  was  ultimately  imple¬ 
mented.  Therefore,  it  is  unclear  whether  the  Navy’s  initial  investment 
in  the  training  of  staff  from  39  SARPs  resulted  in  increased  use  of  the 
CODP  by  service  members  with  CODs.  In  order  to  identify  ways  to 
improve  implementation  of  the  CODP  in  the  future,  and  to  sustain  a 
high  quality  of  care  for  CODs,  a  research  team  from  RAND  was  asked 
to  identify  what  helped  or  hindered  staff  in  implementing  the  CODP 
(i.e.,  facilitators  and  barriers). 

Therefore,  the  aims  of  our  evaluation  were  (1)  to  understand  the 
CODP  training  and  program  goals  and  (2)  to  describe  the  implemen¬ 
tation  of  the  CODP,  including  (a)  describing  the  CODP  training  pro¬ 
gram  and  materials  and  trainee  perceptions  of  the  training,  (b)  iden¬ 
tifying  which  program  elements  were  implemented  and  sustained  at 
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treatment  sites,  (c)  identifying  facilitators  and  barriers  to  successful 
implementation  of  the  CODP,  and  (d)  describing  programs’  capability 
to  provide  integrated  care. 


Organization  of  This  Report 

The  rest  of  this  report  is  organized  as  follows.  Chapter  Two  presents 
more  detail  on  the  CODP  and  its  components;  Chapter  Three  describes 
our  evaluation  methods;  Chapter  Four  presents  a  description  of  the 
survey  participants  and  SARP  characteristics;  Chapter  Five  reviews 
hndings  on  the  perceived  utility  and  quality  of  the  CODP;  Chapter  Six 
describes  findings  on  the  implementation  of  the  CODP;  Chapter  Seven 
presents  the  results  of  our  investigation  into  the  facilitators  and  barri¬ 
ers  to  implementation  of  the  CODP;  Chapter  Eight  describes  findings 
about  SARPs’  capabilities  to  provide  integrated  care  for  CODs;  and 
Chapter  Nine  presents  our  conclusions  and  recommendations. 


CHAPTER  TWO 


The  Design  and  Components  of  the  CODP 


The  Hazelden  Co-Occurring  Disorders  Program 

The  CODP  integrates  several  evidence-based  approaches  to  treatment 
for  CODs,  including  motivational  enhancement  therapy  (MET), 
twelve-step  facilitation  (TSF),  and  cognitive  behavioral  therapy  (CBT). 
It  uses  evidence-based  principles  from  IDDT,  a  program  for  individ¬ 
uals  with  severe  mental  disorders  and  co-occurring  SUDs,  as  well. 
Research  has  shown  that  MET,  TSE,  CBT,  and  IDDT  result  in  signifi¬ 
cant  improvement  in  symptoms  for  clients  with  CODs  (Baker  et  ah, 
2012;  Boden  and  Moos,  2009;  Frisman  et  ah,  2009;  Glasner-Edwards 
et  ah,  2007;  Kushner  et  ah,  2013;  McGovern  et  ah,  2008). 

The  CODP  materials  include  modules  to  guide  clinic  administra¬ 
tors  in  enhancing  their  clinic’s  capability  to  effectively  assess  and  treat 
CODs,  to  educate  providers  about  medication  management-related 
issues,  to  implement  evidence-based  assessment  and  screening  proce¬ 
dures,  and  to  integrate  family  therapy  into  treatment.  Specifically,  the 
CODP  materials  include  the  Clinical  Administrator’s  Guidebook,  five 
curricula  binders,  and  an  educational  DVD  for  patients  and  their  fami¬ 
lies.  The  curricula  binders  are  “Screening  and  Assessment,”  “Integrat¬ 
ing  Combined  Therapies,”  “Cognitive  Behavioral  Therapy,”  “Medica¬ 
tion  Management,”  and  the  “Family  Program.”  Each  binder  includes  a 
clinician  guide,  patient  handouts,  and  a  CD-ROM.  Table  2.1  describes 
each  component. 

In  a  randomized  controlled  trial,  patients  with  severe  PTSD  and 
SUD  who  received  the  CBT  component  showed  significantly  greater 
improvements  in  PTSD  symptoms  and  engagement  in  therapy  than 
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those  who  received  individual  addiction  counseling  (McGovern  et  ah, 
2011).  The  CODP  does  not  include  exposure  therapy,  an  evidence-based 
approach  for  the  treatment  ofPTSD.  Thus,  for  service  members  with  co¬ 
occurring  PTSD,  other  integrated  treatments  that  include  exposure  ther¬ 
apy  may  be  required. 

Table  2.1 

Description  of  CODP  Materials 


CODP  Material 

Description 

Clinical  Administrator's 
Guidebook 

Geared  toward  clinic  administrators 

A  bound  guide  and  CD-ROM  with  reproducible  forms, 
research  articles,  and  other  resources  designed  for 
program/agency  leaders 

Discusses  the  use  of  the  five  curricula  and  the  program 
DVD  to  provide  an  integrated  treatment  program 

Includes  tools  needed  to  assess,  improve,  and  track 
any  addiction  or  mental  health  treatment  program's 
capacity  to  deliver  integrated  treatment  services 

The  CD-ROM  includes  background  research,  Web 
links  to  important  resources,  a  sample  program 
implementation  plan,  and  a  sample  charter  agreement. 

Screening  and  Assessment 

Includes  a  three-ring  binder;  removable,  bound 
clinician's  guide;  and  a  CD-ROM  with  reproducible 
clinician  handouts  and  forms 

Screening  and  assessment  is  the  first  step  in  the 
treatment  process.  This  curriculum  helps  clinicians  learn 
to  detect,  identify,  and  treat  mental  health  disorders  in 
the  context  of  SUDs. 

The  CD-ROM  includes  seven  clinical  assessment  forms 
each  tailored  to  a  specific  mental  health  disorder. 
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Table  2.1 — Continued 


CODP  Material 

Description 

Screening  and  Assessment 
(cont.) 

This  component  covers 

•  the  mental  health  disorders  most  prevalent 
among  patients  with  SUDs 

•  how  to  determine  whether  a  mental  health  symp¬ 
tom  or  disorder  is  substance  induced 

•  how  to  determine  the  patient's  stage  of  motiva¬ 
tion  to  change 

•  the  advantages  and  benefits  of  systematic  screen¬ 
ing  and  how  to  select  a  screening  measure 

•  how  to  decide  which  assessment  method  to  use. 

Integrating  Combined 
Therapies 

Includes  a  three-ring  binder;  removable,  bound 
clinician's  guide;  and  CD-ROM  with  reproducible  patient 
handouts,  fact  sheets,  and  forms 

ICT  uses  a  combination  of  MET,  CBT,  and  TSF.  MET  serves 
to  engage  change,  CBT  to  assist  change,  and  TSF  to 
sustain  change. 

The  clinician's  guide  is  divided  into  two  parts:  (1) 
principles  and  clinician  education  and  (2)  structured 
modules  that  can  be  customized  and  delivered  to 
patients  over  a  two-  to  20-week  period. 

Cognitive  Behavioral 
Therapy 

Includes  a  three-ring  binder;  removable,  bound 
clinician's  guide;  and  a  CD-ROM  with  reproducible 
patient  handouts,  fact  sheets,  and  forms 

The  clinician's  guide  uses  CBT  principles  to  address 
the  most  common  mental  health  disorders  that  are 
presented  in  addiction  treatment  settings,  such  as 
depression,  anxiety,  social  phobia,  and  PTSD. 
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Table  2.1 — Continued 


CODP  Material 


Description 


Medication  Management  Geared  toward  prescribing  providers 

Includes  a  three-ring  binder  with  hard  copies  of 
reproducible  content  and  a  CD-ROM 

The  Medication  Management  curriculum  is  a  valuable 
resource  for  medical  directors  and  clinicians.  It  contains 
vital,  current  information  about  the  complex  issues  of 
medication  management,  including 

•  the  mental  health  symptoms  that  can  accompany 
alcohol  or  drug  intoxication  and  withdrawal 

•  ways  to  collaborate  with  the  patient  to  prepare  a 
medication  plan  and  techniques  to  encourage  a 
patient's  adherence 

•  how  to  use  treatment  strategies  tailored  to  the 
patient's  stage  of  change 

•  ongoing  monitoring  of  mental  health  and  sub¬ 
stance  use  symptoms 

•  best  practices  in  initiating  medication  treatment 
and  providing  continuing  care. 

The  CD-ROM  contains  an  easy-to-use  reference  on 
specific  medications.  The  indications,  administration, 
side  effects,  drug  interactions,  and  other  relevant 
prescribing  information  are  described  for  each 
medication. 


Family  Program  Includes  a  three-ring  binder;  removable,  bound 

clinician's  guide;  and  CD-ROM  with  reproducible  patient 
handouts 

Helps  clinicians  involve  patients  and  family  members  in 
treatment 


DVD:  A  Guide  for  Living 
with  Co-Occurring 
Disorders:  Help  and  Hope 
for  Clients  and  Their 
Families 


This  90-minute  DVD  educates  clients  and  their  family 
members,  reinforces  important  concepts  for  clinicians, 
and  includes  four  chapters: 

•  Introduction  to  co-occurring  disorders 

•  Overview  of  various  psychiatric  disorders 

•  Basics  of  treatment  and  interviews  with  clients 
and  clinicians 

•  Family  component — examines  the  important  role 
of  family  and  friends  in  recovery. 


NOTE:  Adapted  from  Hazelden  Co-Occurring  Disorders  Program,  undated. 
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Navy  Dissemination  of  the  CODP 

Trainings  for  Navy  SARP  staff  first  occurred  between  2008  and  2010, 
beginning  with  addiction  counselors  and  then  shifting  to  mental  health 
practitioners  (Hazelden  Publishing,  2009;  Hazelden  Publishing,  2010). 
SARP  administrators  were  also  invited  to  participate  in  the  training. 
SARP  staff  members  were  not  required  to  express  an  interest  in  treat¬ 
ing  CODs  in  order  to  attend.  Instead,  all  SARP  staff  were  invited  to 
attend,  but  not  required.  Our  contact  at  BUMED  estimated  that  over 
80  percent  of  SARP  staff  attended  the  training,  and  most  expressed  an 
interest  in  treating  CODs.  Civilian  contracted  staff  members  were  not 
eligible  to  attend  due  to  restrictions  on  travel  for  contractors.  Train¬ 
ees  received  CODP  materials  and  attended  an  offsite  training  ranging 
from  five  to  seven  days  at  Hazelden  in  Center  City,  Minnesota.  The 
duration  of  the  training  was  shortened  over  time  to  accommodate  the 
schedules  of  trainees,  and  in  response  to  financial  constraints. 

The  training  included  didactic  and  interactive  sessions  about 
the  treatment  program,  its  rationale,  and  components;  video  presen¬ 
tations  about  the  CODP;  practice  assignments;  the  development  of 
an  “action  plan”  to  implement  the  CODP  at  participants’  SARPs; 
and  live  observation  of  assessment  and  treatment  of  CODs  using  the 
CODP  at  Hazelden.  Per  the  advice  of  the  Navy,  a  brief  overview  of 
the  Family  Program  was  provided,  but  it  was  not  taught  in  detail  since 
most  SARPs  do  not  work  with  families.  Approximately  226  SARP 
staff  members  representing  39  SARPs  were  trained  over  this  period, 
including  SARP  directors,  psychologists,  alcohol  and  drug  counsel¬ 
ors,  psychiatrists,  nurses,  and  chaplains.  Some  of  these  staff  were  active 
duty,  reserve,  or  retired  military  personnel,  and  others  were  civilian 
government  employees  (Hazelden  Publishing,  2010).  After  the  training 
program,  participants  returned  to  their  SARPs  and  were  expected  to 
implement  the  skills  they  had  learned. 


CHAPTER  THREE 


Evaluation  Methods 


In  this  chapter,  we  describe  the  procedures,  measures,  and  analyses 
used  to  understand  the  CODP  training,  its  implementation,  and  har¬ 
riers  and  facilitators  to  implementation.  We  took  a  comprehensive 
approach  to  answering  the  evaluation  questions  using  both  qualitative 
and  quantitative  methods.  Specifically,  we  used  five  research  proce¬ 
dures:  analysis  of  CODP  administrative  data,  key  informant  discus¬ 
sions,  web  survey  of  CODP  trainees,  telephone  discussions  with  SARP 
staff,  and  site  visits  at  selected  SARPs.  In  this  chapter  we  first  describe 
the  CODP  administrative  data  and  use  the  information  gained  from 
those  data  to  describe  the  cohort  of  individuals  who  received  CODP 
training  between  October  2008  and  May  2010.  In  the  next  section  of 
the  chapter,  we  describe  our  methods  for  conducting  a  web  survey  of 
CODP  trainees,  telephone  discussions  with  SARP  staff,  and  the  SARP 
site  visits.  Finally,  we  describe  the  measures  and  analyses,  organized  by 
data  collection  method  (key  informant  discussion,  web  survey,  etc.). 

This  evaluation  was  reviewed  and  approved  by  the  RAND 
Human  Subjects  Protection  Committee.  Subsequently,  administrative 
approvals  were  received  from  the  Defense  Centers  of  Excellence  for 
Psychological  Health  and  Traumatic  Brain  Injury  and  Department  of 
Navy  Human  Research  Protections  Program.  Each  military  treatment 
facility  we  visited  also  reviewed  and  approved  the  protocol.  Participants 
were  fully  informed  of  the  purpose  of  the  study  and  provided  informed 
consent  prior  to  participation. 
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Analysis  of  CODP  Administrative  Data 

Hazelden  routinely  collects  information  from  its  trainees  for  program 
evaluation  purposes.  The  trainees,  as  described  in  Chapter  One,  were 
clinical  staff  working  at  SARPs  in  various  locations  around  the  world 
(see  Table  3.1).  We  reviewed  self-reported  data  collected  by  Hazelden 
that  described  the  226  SARP  staff  who  attended  the  CODP  training 
between  October  2008  and  May  2010. 


Description  of  CODP  Trainees 

According  to  the  self-report  data  obtained  from  Hazelden,  of  the  226 
attendees,  120  were  on  active  duty  at  the  time  of  the  training;  31  were 
civilian  government  employees;  and  57  were  retired  military  person¬ 
nel.  Eighteen  did  not  report  their  military/civilian  status.  Trainees  rep¬ 
resented  a  variety  of  professions,  including  SARP  counselors,  SARP 
directors,  counseling  interns,  social  workers,  psychologists,  nurse  prac¬ 
titioners,  and  chaplains.  They  also  reported  a  range  of  educational 
backgrounds,  ranging  from  “some  college”  to  doctoral  degrees  such 
as  M.D.’s,  Ph.D.’s,  and  Ed.D.’s.  On  average,  trainees  had  been  treating 
clients  with  SUDs  for  6.8  years,  yet  the  number  of  years  of  experience 
in  treating  clients  with  SUDs  varied  widely  (range:  0-40  years).  Train¬ 
ees  worked  for  DoD  for  an  average  of  15.7  years,  with  a  range  of  0.5 
to  43  years.  Trainees  represented  39  SARPs  located  all  over  the  world 
(see  Table  3.1). 


Data  Collection  Methods 

Telephone  Discussions  with  Key  Informants 

At  the  outset  of  the  evaluation,  we  identified  key  informants  with 
unique  knowledge  of  the  initiation  of  the  CODP  at  Navy  SARPs. 
The  first  informant  was  the  Navy  SARP  Director,  who  was  selected 
because  he  was  instrumental  in  initiating  the  CODP  contract  and 
training  program  across  SARP  sites.  The  second  informant  was  the 
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Table  3.1 

SARP  Locations  of  CODP  Trainees  at  the  Time  of  Training 


Trainees 


SARP  Locations 

N 

Percentage 

Totai 

226 

100.0 

Bahrain 

1 

0.4 

Beaufort,  S.C. 

1 

0.4 

Bethesda,  Md. 

5 

2.2 

Bremerton,  Wash. 

9 

4.0 

Brunswick,  Me. 

3 

1.3 

Camp  Lejeune,  N.C. 

17 

7.6 

Camp  Pendleton,  Calif. 

18 

8.0 

Cherry  Point,  N.C. 

2 

0.9 

Corpus  Christi,  Tex. 

5 

2.2 

Dahlgren,  Va. 

1 

0.4 

Fallon,  Nev. 

1 

0.4 

Great  Lakes,  Mich. 

9 

4.0 

Groton,  Conn. 

16 

7.1 

Guam 

4 

1.8 

Henderson  Hall,  Va. 

1 

0.4 

Jacksonville,  Fla. 

15 

6.7 

Kings  Bay,  Ga. 

4 

1.8 

Mayport,  Fla. 

2 

0.9 

Naples,  Italy 

1 

0.4 

Washington  Navy  Yard,  D.C. 

3 

1.3 

Newport,  R.l. 

4 

1.8 

Norfolk/Portsmouth,  Va. 

21 

9.4 

Oak  Harbor,  Wash. 

3 

1.3 
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Table  3.1 — Continued 


SARP  Locations 

N 

Trainees 

Percentage 

Okinawa,  Japan 

8 

3.6 

Parris  Island,  S.C. 

3 

1.3 

Patuxent  River,  Md. 

2 

0.9 

Pearl  Harbor,  Hawaii 

6 

2.7 

Pensacola,  Fla. 

3 

1.3 

Port  Hueneme,  Calif. 

1 

0.4 

San  Diego,  Calif. 

35 

15.6 

Sasebo,  Japan 

3 

1.3 

Sigonella,  Sicily 

1 

0.4 

Spain 

1 

0.4 

Tidewater  Area,  Va. 

1 

0.4 

USS  Abraham  Lincoln 

1 

0.4 

USS  Harry  S.  Truman 

1 

0.4 

Ventura  County,  Calif. 

1 

0.4 

Whidbey  Island,  Wash. 

5 

2.2 

Yokosuka,  Japan 

7 

3.1 

Unknown  SARP 

1 

0.4 

SOURCE:  Hazelden. 
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Hazelden  Manager  of  Special  Projects  (subsequently  referred  to  as  the 
Hazelden  Program  Manager),  who  was  selected  because  of  her  role  as 
the  primary  liaison  with  the  Navy  and  as  the  person  responsible  for 
overseeing  delivery  of  the  training  program  at  Hazelden.  The  Hazelden 
Program  Manager  maintained  contact  with  the  SARP  Director  and 
selected  SARP  leaders  following  the  CODP  training  in  order  to  obtain 
additional  feedback  on  how  CODP  implementation  was  going  at  vari¬ 
ous  SARPs.  Both  informants  were  involved  in  developing  the  plan  for 
this  evaluation  and  were  aware  that  they  would  be  asked  to  participate 
in  telephone  discussions.  We  conducted  individual  telephone  discus¬ 
sions  with  each  informant,  which  lasted  approximately  one  hour.  A 
RAND  researcher  facilitated  the  discussions,  while  a  second  RAND 
researcher  took  notes. 

Web  Survey  of  CODP  Trainees 

Of  the  226  people  who  attended  the  training,  we  administered  a  web 
survey  to  those  who  were  active  duty  military,  reserve  military,  or 
civilian  employees  who  were  retired  military  personnel  or  spouses  of 
a  member  of  the  Navy.  Due  to  regulatory  restrictions,  we  were  unable 
to  administer  the  survey  to  civilians  who  were  not  retired  members  of 
the  military  or  spouses  of  military  personnel.  The  initial  survey  invita¬ 
tion  was  sent  via  email  in  July  2012  to  all  trainees  with  a  valid  email 
address  (136  trainees;  60.2  percent  of  those  who  attended  the  training) 
and  the  survey  was  available  on  the  Internet  for  one  month.  Unfor¬ 
tunately,  BUMED  staff  members  were  unable  to  provide  alternative 
email  addresses  for  those  without  valid  email  addresses.  Reminders 
were  sent  to  those  who  had  not  yet  completed  it  one  and  two  weeks 
after  the  initial  invitation. 

Of  the  136  trainees  successfully  emailed,  58  (42  percent)  accessed 
the  web  survey.  However,  22  of  these  58  individuals  were  ineligible  for 
the  current  study  because  they  were  not  active  duty,  reserve  or  retired 
service  members,  or  spouses  of  service  members  at  the  time  of  the 
survey.  This  may  include  several  who  were  civilian  government  employ¬ 
ees  whom  we  therefore  could  not  include  due  to  regulatory  restrictions, 
but  we  are  unable  to  verify  this.  A  total  of  36  trainees  completed  the 
survey,  resulting  in  an  adjusted  response  rate  of  32  percent  (number 
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of  completes  divided  by  114  eligible  respondents)  and  approximately 
16  percent  of  all  {N  =  226)  trainees.  The  adjusted  response  rate  is  not 
unexpected,  as  lower  response  rates  are  not  unusual  when  surveying 
health  care  providers  (Kellerman  and  Herold,  2001;  VanGeest,  John¬ 
son,  and  Welch,  2007). 

Telephone  Discussions  with  SARP  Staff 

At  the  end  of  the  web  survey,  respondents  were  asked  if  they  were  inter¬ 
ested  in  learning  more  about  participating  in  a  telephone  discussion. 
Those  who  indicated  interest  were  emailed  an  invitation  to  participate 
in  a  telephone  discussion.  The  45-60  minute  discussions  gathered  staff 
perspectives  about  program  utility,  use  of  materials  and  barriers  to  their 
use,  and  sustainability.  A  RAND  researcher  facilitated  the  discussions, 
and  SARP  staff  comments  and  responses  were  transcribed  by  a  RAND 
research  assistant.  Of  the  36  individuals  completing  the  web  survey,  12 
indicated  willingness  to  be  contacted  to  learn  more  about  participat¬ 
ing  in  a  telephone  follow-up  discussion  about  their  experiences  with 
the  CODP.  Three  of  these  12  individuals  subsequently  declined  to  be 
interviewed,  and  two  more  did  not  respond  to  our  attempts  to  contact 
them.  Interviews  were  completed  with  a  total  of  seven  individuals. 

Site  Visits 

Four  SARPs  were  selected  for  site  visits  based  on  findings  from  the 
web  survey  about  SARPs’  level  of  adoption  of  CODP  materials  and 
capability  to  provide  integrated  treatment  for  CODs.  Our  goal  was  to 
maximize  variability  in  SARPs  across  these  dimensions.  We  expected 
that  SARP  size  (e.g.,  number  of  clients  screened/treated)  and  location 
(e.g.,  proximity  to  a  large  Naval  medical  center)  would  be  related  to  the 
implementation  of  the  CODP  and  each  SARP’s  capability  to  provide 
integrated  treatment  to  clients  with  CODs.  Therefore,  we  selected  four 
sites  based  on  the  web  survey  findings,  number  of  clients  screened  and 
treated,  and  proximity  to  a  large  medical  center.  Additional  informa¬ 
tion  about  the  site  selection  methodology  is  available  in  Appendix  A. 

SARP  Directors  at  each  site  were  notified  they  were  selected  for 
a  site  visit  and  provided  with  complete  information  about  the  plans 
for  the  visit.  Two  RAND  researchers  (both  clinical  psychologists) 


Evaluation  Methods  21 


conducted  the  visits.  Visits  included  a  standardized  evaluation  of  the 
site’s  capability  to  provide  integrated  treatment  for  clients  with  co¬ 
occurring  mental  health  and  substance  use  disorders  (one  of  the  goals 
of  the  CODP  training).  Site  visits  involved  observation  of  the  pro¬ 
gram  environment,  brief  discussions  with  SARP  staff,  observation  of 
client  groups,  brief  discussions  with  clients,  and  SARP  medical  record 
review.  SARP  directors  were  asked  to  select  ten  records  from  a  mix  of 
clients  with  and  without  CODs  for  on-site  record  review.  No  identify¬ 
ing  information  was  collected,  and  all  participating  staff  and  clients 
provided  verbal  informed  consent. 


Measures 

Telephone  Discussions  with  Key  Informants 

The  key  informant  discussions  were  guided  by  a  series  of  questions 
designed  to  elicit  the  goals  and  expectations  of  implementing  the  train¬ 
ing  program  in  the  Navy,  perceptions  of  the  program’s  strengths  and 
challenges,  barriers  and  facilitators  of  implementation,  and  perceptions 
of  the  value  of  the  program  (see  Appendix  B:  Key  Informant  Discus¬ 
sion  Guide). 

Web  Survey 

The  survey  (Appendix  C)  assessed  a  number  of  domains  described 
below.  Additional  information  about  the  selected  measures,  including 
their  psychometric  properties  and  scoring,  is  available  in  Appendix  A. 

Demographics  and  Professional  Characteristics  were  collected 
including  age,  sex,  race/ethnicity,  provider  type  (e.g.,  alcohol  and  drug 
counselor,  clinical  psychologist),  primary  professional  role  at  the  SARP 
(clinician,  administrator,  both),  current  SARP  location,  and  whether 
they  were  currently  at  the  same  site  at  the  time  they  received  CODP 
training. 

Characteristics  ofSARPs  were  assessed  using  selected  items  from 
the  National  Survey  of  Substance  Abuse  Treatment  Services  (Depart¬ 
ment  of  Health  and  Human  Services,  2008).  Items  assess  types  of 
services  available  at  the  SARP,  the  proportion  of  clients  who  receive 
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individual  or  group  counseling,  and  how  often  various  therapeutic 
approaches  (e.g.,  CBT)  are  used  at  the  SARP. 

Implementation  and  perceptions  of  the  CODP  materials,  and  barri¬ 
ers  to  implementation  were  measured  using  the  Texas  Christian  Univer¬ 
sity  Workshop  Assessment  Follow-Up  Workshop  Assessment  (TCU- 
WAFU)  (Bartholomew  et  ah,  2007).  The  Workshop  Assessment  is 
a  26 -item  questionnaire  with  acceptable  reliability  and  validity  that 
measures  use  of  and  intentions  to  use  training  materials,  program 
resources  related  to  use,  and  barriers  to  use.  We  used  the  following 
subscales  of  the  TCU-WAFU:  the  Training  subscale  (three  items) 
assesses  participants’  views  on  the  quality  of  the  training;  the  Qual¬ 
ity  and  Utilization  subscale  (four  items)  assesses  training  participants’ 
views  about  the  quality  and  usefulness  of  the  materials;  the  Resources 
and  Skills  subscale  (five  items)  measures  participants’  ability  to  incor¬ 
porate  the  materials  in  their  workplaces;  the  Support  and  Commit¬ 
ment  subscale  (five  items)  identifies  the  extent  to  which  implementing 
the  skills  learned  at  the  training  has  been  supported  by  colleagues  and 
senior  staff,  and  has  been  sustainable;  and  the  Barriers  Encountered 
subscale  (eight  items)  was  used  to  determine  why  participants  did  not 
use  materials  in  their  practice.  All  items  were  rated  on  a  five-point  scale 
(1  =  Disagree  Strongly,  2  =  Disagree,  3  =  Undecided,  4  =  Agree,  5  = 
Agree  Strongly).  Rescaled  average  scores  were  computed  for  the  total 
and  subscales  (Bartholomew  et  ah,  2007).  To  do  this,  the  average  score 
is  computed  and  then  multiplied  by  ten.  Therefore,  scale  scores  can 
range  from  10  to  50. 

To  mQ2iSU.K  frequency  of  use  of  CODP  materials  since  the  training, 
we  developed  nine  items,  which  were  rated  on  a  five-point  scale  (0  = 
Never,  1  =  Once  or  twice,  2  =  A  few  times  a  year,  3  =  Monthly,  4  = 
Weekly).  Respondents  were  asked,  “Which  of  the  following  best  char¬ 
acterizes  how  often  you  used  these  CODP  materials  since  the  training? 
Examples  of  ‘use’  include  using  handouts  or  measures  in  assessment  or 
treatment,  reading  or  reviewing  the  manuals,  or  using  content  to  guide 
a  session.”  Respondents  were  asked  to  rate  the  frequency  with  which 
they  have  used  the  CODP  program  curricula  and  specific  program 
materials  (total  of  nine  items).  We  did  not  assess  trainees’  use  of  the 
Family  Program  component  because  it  was  not  fully  addressed  in  the 
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CODP  training,  as  most  SARPs  do  not  work  with  families.  To  exam¬ 
ine  factors  that  might  he  associated  with  overall  implementation  of  the 
CODP  materials,  we  summed  the  scores  of  all  nine  variables  to  create 
an  “Adoption  of  CODP  Materials”  total  score.  To  evaluate  differential 
implementation  of  screening  and  treatment  materials,  we  computed 
indicators  of  (1)  adoption  of  CODP  screening  materials,  and  (2)  adop¬ 
tion  of  CODP  therapy  materials.  The  screening  variable  was  a  sum  of 
two  items:  frequency  of  use  of  the  CODP  screening  and  assessment 
guide,  and  frequency  of  use  of  the  CODP  screening  and  assessment 
measures.  The  indicator  of  adoption  of  CODP  therapy  materials  was  a 
sum  of  the  items  assessing  frequency  of  use  of  the  Cognitive  Behavioral 
Therapy  and  Integrating  Combined  Therapies  guides. 

The  utility  of  the  CODP  was  assessed  using  an  eight-item  mea¬ 
sure  created  by  Hazelden  to  assess  the  utility  of  the  program.  Psycho¬ 
metric  properties  are  not  available  for  this  measure.  Questions  assess 
self-efhcacy  and  achievement  of  different  program  objectives.  Items  are 
rated  on  a  four-point  scale,  where  4  =  Strongly  agree,  3  =  Somewhat 
agree,  2  =  Somewhat  disagree,  1  =  Strongly  disagree.  The  first  three 
items  were  asked  only  of  respondents  who  reported  serving  in  adminis¬ 
trative  roles.  Because  the  items  assess  diverse  concepts,  these  items  were 
examined  individually. 

Respondents  also  reported  on  the  supervision  and  additional 
training  to  support  implementation  of  the  CODP  they  received  after  the 
Hazelden  training.  Specifically,  respondents  were  asked  whether  they 
had  ever  received  any  clinical  supervision  to  help  perform  assessments 
or  deliver  treatment  using  the  CODP  materials.  If  they  answered 
“yes,”  they  were  also  asked  to  report  which  modality  of  supervision 
was  received  (e.g.,  live  observation,  face-to-face  meetings  with  supervi¬ 
sor,  supervisor  listened  to  audio  recordings  of  sessions),  and  to  estimate 
the  number  of  CODP  supervision  sessions  they  had  received  since  the 
training  (0  =  None,  1  =  1-5,  2  =  6-10,  3  =  11-15,  4  =  16-20,  5  =  21  or 
more).  Finally,  respondents  reported  the  number  of  other  trainings  on 
CODs  they  had  attended  since  the  CODP  training  at  Hazelden. 

Attitudes  about  the  adoption  of  evidence-based  practices  (EBPs)  were 
measured  with  the  15-item  Evidence-Based  Practice  Attitude  Scale, 
which  has  acceptable  reliability  and  validity  (Aarons,  2004).  This  mea- 
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sure  was  selected  since  resistance  to  using  EBPs  could  be  a  barrier  to 
implementation  of  the  CODP.  In  addition  to  the  subscale  scores,  a 
total  score  was  calculated  (14  items),  reflecting  global  attitude  toward 
the  adoption  of  EBPs.  Respondents  were  asked  to  answer  each  question 
based  on  a  0-4  scale  (0  =  Not  at  all;  1  =  To  a  slight  extent;  2  =  To  a 
moderate  extent;  3  =  To  a  great  extent;  4  =  To  a  very  great  extent).  Mean 
scores  were  used  for  analyses  including  attitudes  total  and  subscales. 

Attitudes  about  treating  clients  with  CODs  were  assessed  with  the 
Clinician  Attitudes  Questionnaire  (Hunter  et  ah,  2005).  This  measure 
consists  of  seven  items  (e.g.,  “there  is  little  to  be  done  for  clients  that 
are  mentally  ill”),  which  are  rated  on  a  five-point  scale  (1  =  Strongly 
Disagree,  2  =  Somewhat  Disagree,  3  =  Neither  Disagree  nor  Agree,  4 
=  Somewhat  Agree,  5  =  Strongly  Agree).  A  mean  attitudes  score  was 
computed  (Hunter  et  ah,  2005).  This  measure  was  selected  since  neg¬ 
ative  attitudes  about  individuals  with  mental  health  problems  (e.g., 
“People  with  mental  illness  have  weak  personalities”)  can  serve  as  bar¬ 
riers  to  delivering  effective  treatment  for  CODs. 

Attitudes  about  treating  clients  with  SUDs  were  measured  with  the 
Brief  Substance  Abuse  Attitude  Survey  (Chappel,  Veach,  and  Krug, 
1985).  This  is  a  20-item  survey  derived  from  the  longer  Substance 
Abuse  Attitude  Survey.  These  items  are  rated  on  a  five-point  scale  (1 
=  Strongly  Disagree,  2  =  Disagree,  3  =  Uncertain,  4  =  Agree,  5  = 
Strongly  Agree).  This  measure  was  selected  because  negative  attitudes 
about  clients  with  SUDs  could  be  another  barrier  to  effective  imple¬ 
mentation  of  the  CODP.  This  measure  consists  of  five  subscales,  which 
assess  respondent  beliefs  about  substance  use,  attitudes  about  substance 
abuse  interventions,  common  substance  abuse  stereotypes,  views  on 
the  utility  of  substance  abuse  treatment,  and  moral  issues  around  sub¬ 
stance  abuse. 

Participants  who  were  SARP  administrators  were  asked  to  assess 
the  capability  of  their  SARP  to  provide  services  for  clients  with  CODs 
using  the  Dual  Diagnosis  Capability  in  Addiction  Treatment,  version 
4.0  (DDCAT).  The  DDCAT  is  a  35-item  measure  that  was  developed 
to  assess  capability  to  assess  and  treat  CODs  and  to  guide  addiction 
treatment  providers  in  enhancing  services  for  persons  with  CODs. 
The  reliability  and  validity  of  the  DDCAT  are  acceptable  (McGovern, 
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Matzkin,  and  Giard,  2007).  The  DDCAT  was  presented  and  described 
to  CODP  trainees  at  the  training.  The  DDCAT  was  designed  to  be 
rated  by  trained  observers  who  use  information  from  a  variety  of 
sources  (e.g.,  staff  and  client  interviews,  observation  of  clinic  materi¬ 
als  and  procedures,  chart  review)  to  compute  scores  across  the  seven 
domains.  However,  because  it  was  not  possible  for  the  research  staff  to 
visit  every  site,  we  asked  SARP  administrators  to  rate  their  own  SARPs 
on  the  web  survey.  Self-administered  DDCAT  scores  tend  to  be  posi¬ 
tively  biased  (i.e.,  toward  more  desirable  scores)  by  about  one  point  on 
average  (Lee  and  Cameron,  2009).  We  modified  DDCAT  items  to  be 
appropriate  for  self-report  (as  opposed  to  observer  rated)  and  for  the 
SARP  setting.  We  consulted  with  the  primary  author  of  the  DDCAT 
(Mark  McGovern),  who  reviewed  the  modifications. 

The  DDCAT  includes  35  items  that  are  rated  on  a  Likert  scale. 
The  individual  items  assess  the  extent  to  which  a  site  is  able  to  pro¬ 
vide  COD  treatment.  Example  items  include  “Are  patients  with  mental 
health  disorders  expected  and  welcomed  at  your  SARP?”  and  “To  what 
extent  is  education  about  psychiatric  disorders,  their  treatment,  and 
their  interaction  with  substance  use  &  its  treatment  offered  at  your 
SARP?”  The  ratings  range  from  1  to  5,  where  1  indicates  that  the  SARP 
is  capable  of  providing  little  to  no  treatment  for  mental  health  condi¬ 
tions,  3  indicates  that  the  site  can  provide  some  COD  treatment,  and 
5  indicates  that  the  site  provides  comprehensive  COD  treatment.  The 
DDCAT  consists  of  seven  subscales,  which  assess  a  SARP’s  program 
structure,  program  milieu  (referring  to  whether  patients  with  CODs 
are  welcomed),  assessment  of  CODs,  treatment  of  CODs,  ability  to 
provide  continuity  of  care,  staffing,  and  training.  We  computed  mean 
subscale  scores  for  each  DDCAT  subscale.  Then,  we  averaged  the  mean 
subscale  scores  to  create  a  total  score  for  the  SARP,  which  can  range 
from  1  (Addiction  Only  Services)  to  5  (Dual  Diagnosis  Enhanced)  and 
indicates  the  site’s  overall  level  of  dual  diagnosis  capability. 

Telephone  Discussions  with  SARP  Staff 

Each  discussion  focused  on  participants’  opinions  about  the  program 
materials,  barriers  to  use,  and  sustainability  of  materials  (see  Appendix 
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D:  Staff  Discussion  Guide).  The  discussion  covered  domains  similar  to 
the  web  survey  but  allowed  us  to  collect  more-detailed  information. 

Site  Visits 

Although  we  included  the  DDCAT  in  the  survey,  the  tool  is  predomi¬ 
nantly  used  as  a  structured  site  visit  tool.  The  DDCAT  is  the  most 
frequently  used  tool  to  assess  a  site’s  capability  to  provide  integrated 
care  for  clients  with  COD,  and  has  been  implemented  across  a  number 
of  states  (Substance  Abuse  and  Mental  Health  Services  Administra¬ 
tion,  2011).  The  DDCAT  guides  on-site  data  collection  about  the  pro¬ 
gram’s  services  from  a  variety  of  sources:  (1)  ethnographic  observations 
of  the  milieu,  physical  settings,  and  client  therapy  groups  and  sessions; 

(2)  focused  but  open-ended  discussions  of  agency  directors,  clinical 
supervisors,  clinicians,  support  personnel,  and  clients/consumers;  and 

(3)  review  of  documentation  such  as  medical  records,  program  man¬ 
uals,  brochures,  daily  patient  schedules,  telephone  intake  screening 
forms,  and  other  relevant  materials.  The  characteristics  and  scoring  of 
the  DDCAT  are  described  above. 

Two  RAND  researchers  trained  to  administer  the  DDCAT 
attended  each  site  visit  and  used  information  gathered  during  discus¬ 
sions,  observations,  and  record  review  to  inform  a  dual  diagnosis  capa¬ 
bility  score  using  the  DDCAT.  After  the  site  visit,  each  of  the  two 
DDCAT  raters  independently  scored  the  DDCAT  using  the  published 
scoring  protocol  (Substance  Abuse  and  Mental  Health  Services  Admin¬ 
istration,  2011).  The  raters  then  came  together  to  discuss  their  scoring, 
reconcile  any  discrepancies,  and  determine  final  DDCAT  scores. 


Data  Analyses 

We  integrated  multiple  sources  of  data  to  address  the  evaluation  ques¬ 
tions.  In  this  section,  we  describe  our  approach  to  analyzing  quantita¬ 
tive  data  from  the  web  survey,  qualitative  data  from  discussions  with 
staff  and  key  informants,  and  our  approach  to  presenting  information 
gathered  from  the  site  visits. 
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Web  Survey  of  CODP  Trainees 

We  evaluated  descriptive  statistics  (e.g.,  means,  standard  deviations, 
frequencies)  for  demographic  and  professional  characteristics,  in  addi¬ 
tion  to  SARP  site  characteristics  (these  results  were  presented  earlier 
in  this  chapter).  We  also  used  descriptive  statistics  to  evaluate  trainee 
perceptions  of  the  CODP  training  and  materials.  We  assessed  the  level 
of  adoption  of  CODP  materials  using  descriptive  statistics,  analysis 
of  variance,  and  correlational  analyses  to  examine  relationships  with 
demographic  and  professional  characteristics.  Specihcally,  we  evalu¬ 
ated  whether  gender,  race/ethnicity,  or  type  of  SARP  position  (e.g., 
administrative/clinical  versus  clinical  only)  were  associated  with  adop¬ 
tion  of  the  CODP  materials  (mean  total  adoption  score).  In  addition, 
we  examined  the  correlation  (Pearson,  two-tailed)  between  age  and 
CODP  adoption  (mean  total  adoption  score). 

To  assess  harriers  and  facilitators  to  adoption  of  CODP  materi¬ 
als,  we  used  descriptive  statistics  to  describe  several  variables,  including 
practical  barriers  and  facilitators,  receipt  of  supervision  and  additional 
COD  trainings,  attitudes  toward  EBPs,  and  attitudes  toward  patients 
with  psychological  health  and  substance  use  disorders.  We  also  corre¬ 
lated  the  total  adoption  score,  the  adoption  of  screening  materials  score, 
and  the  adoption  of  treatment  materials  score  with  measures  of  barriers 
and  facilitators  to  implementation  and  the  attitudinal  measures.  We 
had  insufficient  statistical  power  to  conduct  more-complex  tests  due  to 
the  small  sample  size  {n  =  36).  In  addition,  the  time  between  CODP 
training  and  web  survey  participation  ranged  from  two  to  four  years 
and  could  act  as  a  barrier  or  facilitator  to  implementation.  However,  we 
were  unable  to  identify  the  date  each  participant  attended  the  training, 
so  could  not  examine  this  in  our  analyses. 

Telephone  Discussions  with  SARP  Staff 

We  reviewed  the  discussion  notes  in  order  to  identify,  label,  and  group 
responses  relevant  to  each  evaluation  aim.  Staff  responses  with  simi¬ 
lar  concepts  were  grouped  together.  In  addition  to  general  themes,  we 
extracted  staff  responses  pertaining  specifically  to  barriers  and  facilita¬ 
tors  of  implementation  of  the  CODP.  We  provide  a  description  of  these 
results  as  they  relate  to  each  evaluation  aim. 
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Telephone  Discussions  with  Key  Informants 

We  reviewed  the  notes  from  each  of  the  discussions  and  extracted  key 
themes  pertaining  to  each  of  the  evaluation  aims.  Given  that  each  of 
the  two  key  informants  had  a  unique  perspective,  their  responses  were 
not  combined.  We  provide  a  description  of  these  results  as  they  relate 
to  each  evaluation  aim. 

Site  Visits 

Similar  to  our  approach  with  the  telephone  discussion  and  key  infor¬ 
mant  interviews,  we  identified  harriers  and  facilitators  to  implementa¬ 
tion  of  CODP  materials  at  each  site.  In  addition,  we  used  descriptive 
statistics  on  total  DDCAT  score,  suhscores,  and  percentage  of  criteria 
met  to  qualify  for  Addiction  Only  Services,  Dual  Diagnosis  Capable, 
and  Dual  Diagnosis  Enhanced  designations  for  each  site. 


CHAPTER  FOUR 


Description  of  Survey  Participants  and  SARP 
Characteristics 


In  this  chapter,  we  describe  the  SARP  staff  memhers  who  participated 
in  the  weh  survey  and  discussions. 


Survey  Respondents 

Demographic  and  Professional  Characteristics 

A  total  of  36  individuals  completed  the  survey.  Thirteen  were  active 
duty  or  in  an  active  status  (36  percent),  and  23  were  retired  service 
memhers  (64  percent).  Compared  to  nonrespondents,  respondents  were 
more  likely  to  he  retired  service  memhers  (x^(l)  =  6.48,/  =  .01).  Demo¬ 
graphics  of  the  survey  respondents  are  shown  in  Table  4.1.  Respon¬ 
dents  were  predominantly  male,  non-Hispanic  white,  retired  service 
members.  The  majority  of  survey  respondents  (75  percent,  n  =  27)  were 
Navy  alcohol  and  drug  counselors  (ADCs).  Six  were  level  one  ADCs, 
and  21  were  level  two.  The  remaining  respondents  included  certihed 
drug  abuse  counselors/licensed  chemical  dependency  counselors  {n  = 
5),  clinical  social  workers  (ji  =  2),  a  psychiatrist  {n  =  1),  and  a  family 
medicine  physician  {n  =  1).  The  majority  of  respondents  (97  percent) 
reported  serving  in  a  clinical  capacity:  20  respondents  were  counselors/ 
clinicians,  15  were  both  counselors/clinicians  and  SARP  administra¬ 
tors,  and  only  one  was  a  SARP  administrator  and  not  a  counselor/ 
clinician. 
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Table  4.1 

Demographic  Characteristics  of  Survey  Respondents  {N  =  36) 


Characteristic 

N 

Percentage 

Gender 

Male 

26 

72.2 

Female 

10 

27.8 

Race/Ethnicity 

Non-Hispanic,  White 

20 

55.6 

Black 

7 

19.4 

Other® 

4 

11.1 

Hispanic,  White 

3 

8.3 

Age 

Mean 

Minimum  Maximum 

45.8 

30  65 

“  Includes  Asian,  American  Indian/Alaskan  Native,  and  more  than  one  race. 


SARP  Characteristics 

The  majority  of  respondents  (64  percent)  were  located  at  the  same 
SARP  at  which  they  were  located  when  they  participated  in  the  CODP 
training  (Table  4.2).  The  number  of  respondents  from  each  SARP 
ranged  from  one  to  four.  Because  the  number  of  respondents  from  each 
SARP  varied,  it  is  important  to  note  that  these  descriptions  of  SARP 
services  may  slightly  overrepresent  the  characteristics  of  certain  SARPs 
(i.e.,  those  with  more  survey  respondents). 

Substance  abuse-related  services  were  among  the  most  com¬ 
monly  endorsed,  while  mental  health-related  services  were  among  the 
least  likely  to  be  endorsed.  Respondents  reported  on  the  types  of  ser¬ 
vices  provided  at  their  SARP  (Table  4.3).  Nearly  95  percent  of  respon¬ 
dents  indicated  their  SARP  provides  comprehensive  substance  abuse 
assessment  or  diagnosis,  yet  less  than  40  percent  indicated  their  SARP 
provided  comprehensive  mental  health  assessment  or  diagnosis.  Fur¬ 
ther,  only  31  percent  reported  their  SARP  offered  medications  for  psy- 
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Table  4.2 

Current  SARPs  of  Survey  Respondents  {N  =  36) 


SARP  Locations 

N 

Percentage 

Camp  Lejeune,  N.C. 

4 

11.1 

Camp  Pendleton,  Calif. 

3 

8.3 

Dahlgren,  Va. 

1 

2.8 

Groton,  Conn. 

2 

5.6 

Jacksonville,  Fla. 

3 

8.3 

Key  West,  Fla. 

2 

5.6 

Mayport,  Fla. 

1 

2.8 

Mid-South,  Tenn. 

1 

2.8 

Naples,  Italy 

1 

2.8 

Navy  Drug  and  Alcohol  Counselor  School, 

San  Diego,  Calif. 

1 

2.8 

Norfolk/Portsmouth,  Va. 

3 

8.4 

Okinawa,  Japan 

1 

2.8 

Parris  Island,  S.C. 

1 

2.8 

Patuxent  River,  Md. 

1 

2.8 

Pensacola,  Fla. 

2 

5.6 

Point  Loma,  Calif. 

1 

2.8 

San  Diego,  Calif. 

4 

11.1 

Washington  Navy  Yard,  D.C. 

2 

5.6 

Yokosuka,  Japan 

1 

2.8 

Other — BUMED  Detachment  Falls  Church 

1 

2.8 

chiatric  disorders.  While  it  is  possible  that  these  services  are  provided 
elsewhere  at  the  same  MTF,  it  suggests  that  there  may  be  barriers  to 
receiving  care  for  co-occurring  mental  health  problems. 
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Table  4.3 

SARP  Services  Provided  {N  =  36  respondents) 


Services  Provided  by  SARP 

N 

Percentage 

Substance  abuse  education 

36 

100.0 

Screening  for  substance  use 

35 

97.2 

Comprehensive  substance  abuse  assessment  or  diagnosis 

34 

94.4 

Aftercare/continuing  care 

34 

94.4 

Case  management 

34 

94.4 

Discharge  planning 

30 

83.3 

Social  skills  development 

25 

69.4 

Mental  health  education 

23 

63.9 

Screening  for  mental  health 

22 

61.1 

Interim  services  for  clients  when  admission  is  not  possible 

20 

55.6 

Self-help  groups 

20 

55.6 

Smoking  cessation  medications 

17 

47.2 

Mentoring/peer  support 

16 

44.4 

Comprehensive  mental  health  assessment  or  diagnosis 

14 

38.9 

Medication-assisted  therapy 

13 

36.1 

Outreach  to  persons  in  the  community  who  might  need 
treatment 

12 

33.3 

Medications  for  psychiatric  disorders 

11 

30.6 

Respondents  also  reported  on  the  treatment  approaches  provided 
at  their  SARP  (Table  4.4).  Results  suggest  that  group  counseling  was 
provided  more  frequently  than  individual  counseling.  Yet  nearly  28 
percent  of  respondents  report  that  less  than  one-half  of  their  patients 
receive  individual  therapy,  and  11  percent  reported  less  than  one-half 
of  their  patients  received  group  therapy.  This  suggests  that  some  ser¬ 
vice  members  seen  at  SARP  do  not  receive  any  individual  or  group 
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psychotherapy.  The  reasons  for  this  are  unclear,  hut  possible  explana¬ 
tions  include  that  some  service  members  are  screened  at  one  SARP 
and  then  referred  to  another  (perhaps  due  to  capacity  or  geographi¬ 
cal  issues),  the  service  member  may  be  receiving  other  treatment  (e.g., 
medication)  that  is  working  well,  the  service  member  may  have  refused 
counseling,  or  there  may  be  a  waitlist  for  counseling.  Nearly  all  respon¬ 
dents  reported  that  substance  use  counseling  and  relapse  prevention 
were  delivered,  yet  fewer  endorsed  that  other  approaches  were  delivered 
often.  For  each  of  the  evidence-based  therapeutic  approaches  included 
in  the  CODP,  over  50  percent  reported  that  they  were  delivered  often 
or  always.  Given  that  the  majority  of  respondents  indicated  that  ser¬ 
vice  members  at  their  SARP  receive  either  group  or  individual  therapy, 
these  results  suggest  that  there  may  be  opportunities  to  increase  the  fre¬ 
quency  with  which  evidence-based  interventions  are  delivered.  Finally, 
31  percent  of  respondents  indicated  that  their  SARP  offered  a  special 
program  for  clients  with  COD.  This  is  consistent  with  the  key  infor¬ 
mant  interview  with  the  SARP  Director,  which  suggested  that  particu¬ 
lar  SARPs  specialize  in  treating  service  members  with  COD  and  that 
often  service  members  with  COD  can  be  transported  to  these  sites  to 
receive  COD  treatment. 


Table  4.4 


Treatment  Approaches  Provided  at  Respondents'  SARPs  {N  = 

36) 

Treatment  Modality 

N 

Percentage 

Individual  counseling 

Received  by  more  than  75  percent  of  clients 

22 

61.1 

Received  by  51  percent  to  75  percent  of  clients 

4 

11.1 

Received  by  25  percent  to  50  percent  of  clients 

5 

13.9 

Received  by  25  percent  or  less  of  clients 

3 

8.3 

Not  offered 

2 

5.6 

Group  counseling 
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Table  4.4 — Continued 

Treatment  Modality 

N 

Percentage 

Received  by  more  than  75  percent  of  clients 

31 

86.1 

Received  by  51  percent  to  75  percent  of  clients 

1 

2.8 

Received  by  25  percent  to  50  percent  of  clients 

2 

5.6 

Received  by  25  percent  or  less  of  clients 

1 

2.8 

Not  offered 

1 

2.8 

Substance  use  counseling 

Always  or  often 

33 

91.7 

Sometimes 

0 

0.0 

Never  or  rarely 

1 

2.8 

Not  familiar  with  this  approach 

1 

2.8 

Twelve-step  facilitation 

Always  or  often 

20 

55.6 

Sometimes 

11 

30.6 

Never  or  rarely 

4 

11.1 

Not  familiar  with  this  approach 

1 

2.8 

Supportive  counseling 

Always  or  often 

25 

69.4 

Sometimes 

7 

19.4 

Never  or  rarely 

2 

5.6 

Not  familiar  with  this  approach 

1 

2.8 

Cognitive  behavioral  therapy 

Always  or  often 

23 

63.9 

Sometimes 

8 

22.2 

Never  or  rarely 

4 

11.1 

Not  familiar  with  this  approach 

1 

2.8 
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Table  4.4 — Continued 


Treatment  Modality 

N 

Percentage 

Contingency  management  or  motivational  incentives 

Always  or  often 

10 

27.8 

Sometimes 

16 

44.4 

Never  or  rarely 

4 

11.1 

Not  familiar  with  this  approach 

5 

13.9 

Motivational  interviewing 

Always  or  often 

22 

61.1 

Sometimes 

12 

33.3 

Never  or  rarely 

1 

2.8 

Not  familiar  with  this  approach 

0 

0.0 

Combined  cognitive  behavioral  therapy,  motivational 
interviewing,  twelve-step  facilitation  for  treating  co¬ 
occurring  disorders 

Always  or  often 

18 

50.0 

Sometimes 

10 

27.8 

Never  or  rarely 

8 

22.2 

Not  familiar  with  this  approach 

0 

0.0 

Relapse  prevention 

Always  or  often 

30 

83.3 

Sometimes 

5 

13.9 

Never  or  rarely 

1 

2.8 

Not  familiar  with  this  approach 

0 

0.0 

Special  co-occurring  disorders  program 

Yes 

11 

31 

No 

24 

69 
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SARP  Staff  Interviewees 

Interviews  were  completed  with  a  total  of  seven  individuals.  These 
included  four  males  and  three  females,  and  two  active  duty/active  status 
and  five  retired  service  members.  Four  interviewees  were  non-Hispanic 
white,  two  were  Hispanic  white,  and  one  was  black.  One  interviewee 
was  a  certified  drug  abuse  counselor,  and  the  other  six  were  Navy  level 
two  alcohol  and  drug  counselors.  Three  of  the  seven  interviewees  were 
counselors/clinicians,  and  four  were  both  administrators  and  counsel¬ 
ors/clinicians.  The  mean  age  of  interviewees  was  44.2  years  (SD  =  8.7). 

SARPs  Selected  for  Site  Visits 

Details  on  how  we  selected  SARPs  for  site  visits  are  contained  in 
Appendix  A.  Table  4.5  shows  how  the  four  selected  sites  represented 
use  of  CODP  materials,  capability  to  provide  care  for  CODs  (based  on 
web  survey  DDCAT  data),  and  SARP  size.  Site  visits  to  three  of  the 
four  sites  were  completed  October  to  November  2012.  Unfortunately, 
due  to  unforeseen  circumstances,  we  were  not  able  to  conduct  a  visit 
at  Site  B. 

The  following  are  characteristics  of  the  sites  we  selected  based  on 
these  criteria: 

•  Site  A:  Self-rated  low  CODP  adopters,  no  web  survey  DDCAT; 
over  2,000  clients  screened  in  2011 

•  Site  B:  Self-rated  high  CODP  adopters.  Dual  Diagnosis  Capable; 
over  1,000  clients  screened  in  2011 

•  Site  C:  Self-rated  low  CODP  adopters.  Addiction  Only  services; 
over  200  clients  screened  in  2011 

•  Site  D:  Self-rated  high  CODP  adopters.  Dual  Diagnosis  Capable; 
under  100  clients  screened  in  2011. 
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Table  4.5 

SARPs  Selected  for  Site  Visits 


SARP  frequency  of  use  of  CODP  materials  (coincides  with  Dual 
Diagnosis  Capability) 

SARP  Size 

Low 

High 

Large 

Site  A 

Site  B 

Small 

Site  C 

Site  D 

CHAPTER  FIVE 


Perceived  Utility  and  Quality  of  the  CODP 


In  this  chapter,  we  describe  CODP  trainee  and  key  informant  per¬ 
ceptions  of  the  Hazelden  training  program  and  materials.  We  include 
results  from  the  survey  of  SARP  staff  members  who  participated  in  the 
CODP  training,  discussions  with  SARP  staff,  and  discussions  with  key 
informants. 


Perceived  Utility  and  Quality  of  the  CODP  Training  and 
Materials 

To  describe  the  implementation  of  the  CODP,  we  sought  to  under¬ 
stand  key  informant  and  trainee  (SARP  staff)  views  about  the  quality 
and  usefulness  of  the  CODP  training  and  materials.  We  examined 
quantitative  data  from  the  survey  and  qualitative  data  from  SARP  staff 
and  key  informant  interviews. 

CODP  Trainees  Reported  Positive  Perceptions  of  the  Training 

Respondents  generally  agreed  that  the  practice  sessions,  instructions, 
and  examples  from  the  CODP  training  provided  them  the  confidence 
and  ability  needed  to  use  the  materials,  and  to  train  others  to  use  the 
materials  (Workshop  Assessment  Training  score  mean  =  37.7  [possible 
scale  range  10-50],  SD  =  6.3).  When  administrators  {N  =  16)  were 
asked  to  assess  how  well  the  CODP  training  enhanced  their  ability 
to  evaluate  their  organizational  capabilities  and  challenges,  they  typi¬ 
cally  agreed  that  they  were  better  able  to  assess  areas  of  organizational 
effectiveness  (M  =  3.0,  SD  =  0.9,  where  4  indicated  strongly  agree 
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and  1  indicated  strongly  disagree),  to  interpret  guides  for  assessing 
organizational  capacity  for  treating  CODs  such  as  the  DDCAT  (M 
=  3.1,  SD  =  0.8),  and  to  identify  challenges  to  providing  integrated 
services  (M  =  3.2,  SD  =  0.8).  Respondents  generally  agreed  that  as  a 
result  of  attending  the  CODP  training,  they  were  better  able  to  use 
protocol- driven  screening  methods  (M  =  3.3,  SD  =  0.8);  to  integrate 
MET,  CBT,  and  TSF  (M  =  3.3,  SD  =  0.8);  to  use  CBT  for  mental 
health  disorders  (M  =  3.2,  SD  =  0.8);  to  examine  military-relevant  dis¬ 
orders  such  as  PTSD  and  TBI  (M  =  3.2,  SD  =  0.8);  and  to  convey 
information  on  issues  such  as  medication  compliance  and  stigma 
(M  =  3.0,  SD  =  0.8).  Figure  5.1  shows  the  distribution  for  each  item. 
The  program  objectives  for  which  there  was  the  most  disagreement 
were  “assess  organizational  effectiveness”  and  “examine  PTSD.” 


Figure  5.1 

Perceptions  of  CODP  Training  Achieving  Program  Objectives  {N  =  36) 
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*These  items  were  answered  only  by  the  16  respondents  who  held  administra¬ 
tive  positions. 
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CODP  Trainees  Reported  that  the  CODP  Met  Program  and  Personal 
Goals 

SARP  staff  members  were  asked  to  talk  about  ways  in  which  the  CODP 
met  its  goal  “to  enhance  Navy  Alcohol  and  Other  Drug  Counselors’ 
ability  to  assess  and  treat  individuals  who  have  co-occurring  disorders.” 
Many  SARP  staff  reported  that  participation  in  the  CODP  training 
improved  their  understanding  of  CODs  and  how  to  treat  them  and 
that  they  were  able  to  incorporate  at  least  some  of  the  CODP  screening 
and  treatment  procedures  into  their  practice.  SARP  staff  reported  that 
the  training  had  its  limits.  For  example,  one  interviewee  noted  that  the 
training  did  not  prepare  him/her  for  how  to  address  more  severe  and 
rare  CODs  such  as  schizophrenia.  Another  noted  that  assessment  and 
treatment  of  individuals  with  CODs  could  be  improved  if  trainees  had 
been  provided  instruction  on  how  to  tailor  or  adapt  the  training  to  the 
population  of  patients  typically  seen  at  their  SARPs. 

We  also  asked  staff  to  discuss  their  personal  goals  for  attend¬ 
ing  the  CODP.  Most  responded  that  they  hoped  to  learn  more  about 
assessment  and  treatment  of  CODs.  Some  added  that  they  were  eager 
to  see  how  the  CODP  could  be  integrated  with  other  programs  used  at 
their  SARPs,  and  to  provide  some  relief  to  the  mental  health  system  in 
addressing  mental  health  needs  of  post-deployment  service  members. 
One  interviewee  reported  very  low  expectations  for  the  program  ini¬ 
tially,  but  was  pleasantly  surprised  by  its  quality.  The  majority  of  inter¬ 
viewees  said  the  CODP  did  meet  their  personal  goals.  Many  noted 
that  the  program  and  its  materials  were  explained  well,  and  that  they 
appreciated  being  provided  the  materials  to  use  after  returning  from 
the  training. 

Key  Informants  Believed  the  Training  Was  Successful  and  Met  Goals 

The  Hazelden  Program  Manager  noted  that  the  educational  back¬ 
grounds  of  participants  in  the  CODP  training  varied  widely.  For  exam¬ 
ple,  she  said  that  some  had  just  completed  the  Navy  Drug  and  Alco¬ 
hol  Counselor  School,  and  others  had  been  practicing  for  many  years, 
or  had  master’s  degrees  in  social  work,  counseling,  or  related  mental 
health  fields.  Although  this  might  have  been  a  potential  obstacle  to 
effective  training,  this  informant  said  the  training  was  a  success.  For 
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example,  she  said  she  could  not  rememher  anyone  saying  the  training 
was  not  worth  his/her  time.  Instead,  she  observed  that  everyone  left  the 
training  “rejuvenated”  and  “pumped  up”;  that  there  was  a  very  “posi¬ 
tive  energy”  around  the  training.  Her  impression  was  that  the  trainees 
were  excited  to  return  to  their  SARPs  and  begin  implementing  the  rel¬ 
evant  components  of  the  CODP. 

The  SARP  Director  stated  that  the  benefits  of  the  program  were 
“absolutely”  worth  the  costs  and  that  they  have  already  completed  addi¬ 
tional  CODP  trainings  for  newer  counselors.  He  noted  that  BUMED 
is  limited  in  its  ability  to  send  contract  SARP  staff  to  trainings,  but 
that  he  wishes  he  were  able  to  send  more  of  them.  At  some  SARPs,  a 
large  proportion  of  counselors  are  contractors.  The  SARP  Director  said 
he  believes  the  CODP  has  led  to  an  expansion  and  enhancement  of 
patient  care  in  the  SARPs  due  to  the  provision  of  services  for  those  with 
CODs,  including  those  with  complex  problems  and  who  are  high  risk. 
He  noted  that  a  limitation  of  the  CODP  training  is  that  the  curriculum 
uses  language  tailored  for  civilian  rather  than  military  populations. 


Trainees  and  Key  Informants  Provided  Suggestions  for 
Improving  the  Effectiveness  of  the  CODP  Training 

When  asked  to  discuss  how  the  CODP  did  or  did  not  expand  the  capa¬ 
bility  of  their  SARP  to  treat  clients  with  CODs,  responses  from  SARP 
staff  interviewees  were  again  mostly  positive.  Interviewees  generally 
believed  they  gained  knowledge  about  CODs,  their  assessment  and 
treatment,  and  that  this  helped  to  better  identify  and  treat  these  prob¬ 
lems.  Some  limitations  were  noted  as  well.  For  instance,  an  interviewee 
reiterated  that  it  would  have  been  helpful  to  learn  how  to  adapt  the  pro¬ 
gram  for  his/her  specific  patient  population.  Several  interviewees  also 
stated  that  while  they  believe  the  CODP  could  substantially  improve 
patient  care,  lack  of  confidence  and  buy-in  from  SARP  administrators, 
the  chain  of  command,  and/or  counselors  prevented  it  from  expand¬ 
ing  their  SARP’s  capability  to  address  CODs.  Therefore,  they  recom¬ 
mended  that  more  of  these  “key  players”  attend  the  training. 


Perceived  Utility  and  Quality  of  the  CODP  43 


SARP  Staff  interviewees  had  a  number  of  additional  suggestions 
for  how  to  enhance  the  effectiveness  of  CODP  training.  Many  said 
it  would  be  easier  to  use  the  skills  learned  in  the  training  if  all  staff, 
including  civilian  staff,  were  trained.  With  regard  to  content,  the  fol¬ 
lowing  were  mentioned  as  potentially  useful  additions:  increased  time 
for  observing  practitioners  at  Hazelden  during  the  training,  specihc 
training  on  how  to  tailor  the  program  to  certain  patient  populations 
(e.g.,  combat  exposed,  inpatient),  and  information  on  how  to  modify 
the  treatment  approaches  for  brief  therapy.  Several  interviewees  also 
said  that  more  follow-up  training  and  supervision  would  help  to 
improve  fidelity  to  the  program  after  the  training. 

When  asked  what  she  would  do  differently  if  she  were  to  deliver 
the  CODP  training  to  SARP  staff  again,  the  Hazelden  Program  Man¬ 
ager  made  suggestions  consistent  with  those  of  the  interviewed  SARP 
staff.  She  recommended  that  (if  funding  could  be  made  available)  other 
staff  attend  (e.g.,  additional  mental  health  staff).  Further,  she  suggested 
that  to  facilitate  systemwide  understanding  and  implementation  of  the 
CODP,  BUMED  widely  communicate  what  the  CODP  is  about,  the 
rationale  for  implementation  of  the  CODP,  and  that  it  will  be  the  new 
standard  of  care  for  clients  with  CODs.  The  Hazelden  Program  Man¬ 
ager  added  that  in  the  future,  she  would  like  to  add  more  training 
on  staff  support  and  clinical  supervision  for  the  SARP  directors.  She 
also  reported  hearing  from  several  trainees  that  the  Navy’s  Drug  and 
Alcohol  Program  Advisors  (DAPAs)  would  benefit  from  the  training. 
DAPAs  inform  their  commanding  officers  about  substance  use  within 
the  command  and  make  recommendations  about  how  to  address  iden¬ 
tified  substance  use  problems. 


CHAPTER  SIX 


Implementation  of  the  CODP  at  SARPs 
Post-Training 


In  this  chapter,  we  focus  on  identifying  which  program  elements  were 
implemented  and  sustained  at  treatment  sites.  We  include  results  from 
the  survey  of  SARP  staff  members  who  participated  in  the  CODP  train¬ 
ing,  discussions  with  SARP  staff,  and  discussions  with  key  informants. 


implementation  of  the  CODP  Materials  Varied  Across 
Component 

Respondents  reported  that  certain  CODP  materials  were  used  regu¬ 
larly  (Figure  6.1).  The  highest  mean  level  of  usage  was  reported  for  the 
ICT  binder  (M  =  2.5,  SD  =  1.4,  with  response  options  ranging  from  0 
for  never  to  4  for  weekly),  and  the  lowest  for  the  Medication  Manage¬ 
ment  binder  (M  =  1.1,  SD  =  1.3).  Figure  6.2  illustrates  respondents’ 
use  of  CODP  supporting  materials  (i.e.,  clinician’s  guide,  workbook 
fact  sheets,  workbook  handouts,  and  screening/assessment  measures). 
Respondents  indicated  they  used  these  materials  at  similar  frequencies. 
The  screening/assessment  measures  were  used  most  often  (M  =  2.4,  SD 
=  1.4),  and  the  clinician’s  guide  was  used  least  often  (M  =  1.9,  SD  = 
1.3).  Mean  frequency  of  use  of  screening  materials  overall  (i.e..  Screen¬ 
ing  and  Assessment  binder,  screening/assessment  measures)  indicated 
that  respondents  used  these  materials  a  few  times  a  year,  on  average  (M 
=  2.3,  SD=1.4).  Respondents  reported  using  the  treatment  materials 
(i.e.,  CBT,  ICT  binders)  with  a  similar  frequency  (M  =  2.4,  SD  =  1.4). 
Overall  adoption  of  the  CODP  materials  was  not  signihcantly  associ¬ 
ated  with  gender  (F(l)  =  0.06,  ns),  race/ethnicity  (white/non-Hispanic 
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Figure  6.1 

Reported  Use  of  the  CODP  Materials,  Binders  {N  =  36) 
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versus  other  race/ethnicity)  (F(l)=1.57,  ns),  or  type  of  SARP  position 
(administrative/clinical  or  clinical  only)  (F(l)=0.06,  m). 


Figure  6.2 

Reported  Use  of  CODP  Materials,  Supporting  Materials  (N  =  36) 
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Use  of  Specific  CODP  Materials 
Administrator's  Guidebook 

Although  only  four  of  the  seven  interviewees  reported  being  SARP 
administrators,  all  were  asked  about  the  administrator’s  guidebook, 
as  others  may  have  held  administrative  responsibilities  in  the  past. 
Few  interviewees  reported  implementing  the  administrator’s  guide¬ 
book.  Those  who  had  referenced  the  guidebook  reported  that  they  had 
looked  at  it  once  or  twice  since  the  training,  as  a  reference  tool.  None 
of  the  interviewees  said  they  hnd  the  guidebook  to  be  very  useful.  In 
addition,  most  SARP  staff  interviewed  had  not  completed  a  DDCAT 
since  the  CODP  training.  One  had  completed  the  DDCAT  after  the 
training  and  said  it  was  “valuable,”  but  that  it  “probably  needs  to  be 
done  again.” 

Screening  and  Assessment  Curricuium 

Most  interviewees  reported  that  they  use  parts  of  the  CODP  screen¬ 
ing  and  assessment  curriculum  when  they  are  relevant  or  applicable 
to  a  particular  patient  or  patient  population.  For  example,  one  inter¬ 
viewee  said  s/he  does  not  typically  use  these  materials  for  patients  who 
have  already  been  seen  by  mental  health  professionals.  One  interviewee 
stated  that  s/he  does  not  use  these  materials  at  all  because  his/her  SARP 
uses  the  Navy’s  existing  screening  tools,  which  are  “comprehensive.” 
Those  who  did  use  the  curriculum  reported  that  the  materials  were 
helpful  for  tracking  patient  progress  and  for  communicating  with  other 
providers. 

Integrating  Combined  Therapies  Curriculum 

Most  interviewees  were  not  using  the  ICT  curriculum  at  the  time  of 
their  interviews.  They  indicated  that  they  rely  on  mental  health  profes¬ 
sionals  to  provide  this  type  of  treatment.  Two  interviewees  said  they 
would  like  to  use  it,  but  that  more  counselor  training  and  supervision 
is  needed.  Another  interviewee  said  s/he  had  adapted  components  of 
the  curriculum  in  designing  a  treatment  program  in  collaboration  with 
mental  health.  Only  one  interviewee  had  implemented  large  portions  of 
the  ICT  curriculum.  This  individual  was  using  many  of  the  handouts 
throughout  the  treatment  process,  beginning  with  the  first  (intake)  ses¬ 
sion.  In  sum,  most  interviewees  did  not  feel  comfortable  using  the  ICT 
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curriculum.  As  suggested  by  some  interviewees,  implementation  of  the 
ICT  curriculum  may  be  improved  with  additional  training. 

Cognitive  Behavioral  Therapy  Curriculum 

The  CBT  curriculum  was  implemented  by  several,  but  not  all,  inter¬ 
viewees.  Those  who  were  using  the  curriculum  found  it  to  be  useful  for 
issues  like  depression  and  anxiety,  and  for  helping  patients  understand 
the  relationships  between  their  environments,  thoughts,  feelings,  and 
behaviors.  Again,  most  users  were  selecting  particular  portions  of  the 
curriculum  that  they  believed  ht  best  for  their  clients  and  setting.  One 
interviewee  stated  that  s/he  did  not  use  the  CBT  curriculum  because 
s/he  did  not  want  to  “overwhelm  the  patient  with  paperwork.”  Those 
using  the  curriculum  said  they  felt  the  materials  were  clear,  logical, 
and  user  friendly.  One  interviewee  said  s/he  appreciated  the  “ability  to 
move  freely  throughout  the  curriculum  and  only  use  what  is  needed.” 

Medication  Management 

In  general,  interviewees  did  not  report  using  the  medication  manage¬ 
ment  curriculum  on  a  regular  basis.  Interviewees  generally  viewed 
the  medication  management  curriculum  as  not  relevant  for  their  job 
responsibilities.  Some  did  not  use  the  curriculum  at  all,  and  said  this 
was  because  another  provider  was  responsible  for  medication  manage¬ 
ment.  Others  found  the  curriculum  helpful  as  a  reference  guide  to 
understand  what  patients  were  taking.  Future  CODP  trainings  could 
modify  training  on  medication  management  to  be  in  depth  for  pre¬ 
scribing  providers,  and  simply  a  general  overview  for  others. 


SARP  Staff  Interviewee  Perspectives  on  the 
Implementation  of  the  CODP 

Most  participants  in  the  CODP  training  completed  an  “action  plan” 
at  the  end  of  the  training,  where  they  identified  specific  actions  they 
planned  to  take  to  improve  their  SARP’s  dual  diagnosis  capability 
within  three  areas:  the  intake  and  assessment  phase,  the  care  phase, 
and  the  continuing  care  phase.  Interviewees  who  had  completed  action 
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plans  were  asked  about  their  implementation  of  the  action  items  they 
listed  in  each  phase. 

Intake  and  Assessment  Phase 

Most  SARP  staff  interviewees  reported  that  they  had  implemented  their 
action  items  in  the  intake  and  assessment  phase.  Interviewees  reported 
that  their  SARPs  use  the  CODP  screening  and  assessment  tools  and 
procedures  but  have  adapted  them  for  their  particular  population  and 
setting.  For  example,  one  interviewee  said,  “we  incorporated  the  sug¬ 
gestions  from  Hazelden  that  we  felt  would  be  applicable  to  our  popula¬ 
tion.”  Another  said  “we  integrated  parts  of  the  [CODP]  screener  with 
other  measures  from  other  programs.”  One  interviewee  reported  that 
s/he  did  not  implement  the  screening  and  assessment  tools  as  planned 
because  of  patients’  being  largely  referred  from  mental  health,  where 
they  have  already  had  a  recent  assessment  of  mental  health  problems. 

Care  Phase 

Action  items  listed  in  the  care  phase  were  less  consistently  implemented. 
Most  interviewees  had  implemented  some,  but  not  all,  of  their  plans. 
One  interviewee  said  that  despite  his/her  goals,  the  design  and  struc¬ 
ture  of  the  COD  treatment  program  has  been  a  “work  in  progress.” 
S/he  suggested  that  because  the  SARP  and  the  mental  health  depart¬ 
ment  are  working  together  to  design  the  program,  it  has  sometimes 
been  difficult  to  decide  upon  policies  that  are  mutually  agreeable. 
Another  said  s/he  implemented  some,  but  not  all,  action  items,  based 
on  what  was  most  useful  and  necessary.  Interviewees  noted  that  some 
of  the  things  that  could  make  these  changes  within  the  care  phase 
sustainable  include  counselor  training  and  program  structure.  Specifi¬ 
cally,  if  new  counselors  are  trained  in  the  CODP  and  current  coun¬ 
selors  are  provided  “booster”  trainings  and  supervision,  the  program 
is  more  likely  to  be  sustainable.  One  interviewee  added  that  his/her 
SARP  was  changing  in  structure,  so  that  it  will  soon  include  an  inpa¬ 
tient  program.  This  will  result  in  a  different  patient  population  and 
treatment  program,  which  could  interfere  with  the  sustainability  of  the 
CODP  within  that  SARP. 


50  Improving  Care  for  Co-Occurring  Psychological  and  Substance  Use  Disorders 


Continuing  Care  Phase 

A  mix  of  experiences  was  reported  by  interviewees  when  asked  about 
the  implementation  of  their  action  items  relevant  to  the  continuing 
care  phase.  One  interviewee  reported  that  s/he  was  unable  to  imple¬ 
ment  these  items  because  the  “command  was  not  on  board.”  Others 
indicated  that  they  were  able  to  implement  their  action  items,  after 
adapting  the  CODP  materials  and  curriculum  to  work  with  their 
patients  and  system  of  care. 


Key-Informant  Perspectives  Were  Similar  to  SARP  Staff 
Perspectives 

The  Hazelden  Program  Manager  indicated  that  she  has  received  feed¬ 
back  primarily  from  the  larger  SARPs  since  delivery  of  the  CODP  train¬ 
ings.  She  reported  implementation  of  the  CODP  has  varied  depend¬ 
ing  on  each  SARP’s  level  of  care.  For  example,  she  said  some  SARPs 
provide  screening  and  continuing  care,  but  not  treatment.  She  added 
that  most  SARPs  do  not  provide  medication  management.  Therefore, 
there  are  likely  to  be  particular  portions  of  the  CODP  that  are  more  or 
less  relevant  depending  on  the  SARP.  The  Hazelden  Program  Manager 
said,  from  the  feedback  she  has  received,  the  counselors  who  attended 
the  training  are  “doing  integrated  treatment  to  the  best  of  their  abil¬ 
ity.”  She  noted  that  they  have  taken  the  tools  and  adapted  them  for 
their  clients.  The  Hazelden  Program  Manager  acknowledged  that  the 
family  component  of  the  CODP  has  not  been  implemented  because 
few  SARPs  provide  family  programs. 

Consistent  with  the  Hazelden  Program  Manager’s  report,  the 
SARP  Director  also  believed  the  CODP  materials  and  training  were 
being  implemented  and  perceived  as  useful  by  SARP  staff.  He  noted 
that  use  of  the  materials  likely  varied  depending  on  staff  motivation. 
The  SARP  Director  reported  that  he  did  not  have  any  formal  mecha¬ 
nism  for  monitoring  SARP  staff  implementation  of  the  CODP. 


CHAPTER  SEVEN 


Facilitators  and  Barriers  to  CODP  Implementation 


In  this  chapter,  we  identify  facilitators  and  harriers  to  successful  imple¬ 
mentation  of  the  CODP.  In  addition  to  understanding  overall  levels  of 
implementation  of  the  CODP  materials,  we  sought  to  understand  why 
the  CODP,  and  particular  components  of  it,  were  more  or  less  likely 
to  he  implemented  than  others.  We  include  results  from  the  survey  of 
SARP  staff  members  who  participated  in  the  CODP  training,  discus¬ 
sions  with  SARP  staff,  discussions  with  key  informants,  and  the  site 
visits. 


Several  Factors  Facilitated  Adoption  of  CODP  Materials 

Survey  respondents  generally  agreed  that  the  CODP  materials  were 
useful  and  of  high  quality  (Workshop  Assessment  Quality  and  Uti¬ 
lization  score:  M  =  41.9  [possible  scale  range  10-50],  SD  =  7.1).  In 
addition,  respondents  believed  their  SARPs  provide  sufficient  staffing, 
resources,  and  time,  and  that  SARP  counselors  have  adequate  train¬ 
ing  and  have  implemented  the  materials  effectively  (Workshop  Assess¬ 
ment  Resources  and  Skills  score:  M  =  41.3  [possible  scale  range  10-50], 
SD  =  11.6).  On  average,  respondents  also  felt  that  their  SARP  directors, 
supervisors,  and  fellow  staff  supported  and  encouraged  one  another  to 
use  the  CODP  materials  (Workshop  Assessment  Support  and  Com¬ 
mitment  score:  M  =  37.0  [possible  scale  range  10-50],  SD  =  8.1). 

In  our  analysis  of  the  web  survey  data,  we  sought  to  identify 
factors  associated  with  higher  adoption  of  CODP  materials.  In  addi¬ 
tion  to  overall  adoption  of  CODP  materials,  we  also  evaluated  factors 
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associated  with  use  of  CODP  screening  and  assessment  materials  and 
CODP  therapy  materials.  These  two  suhscales  were  highly  correlated 
(r  =  0.58)  hut  still  suggested  that  evaluating  these  portions  of  CODP 
use  separately  was  warranted.  We  examined  the  associations  between 
CODP  adoption  and  harriers  and  facilitators  and  attitudinal  measures 
(Table  7.1).  Higher  overall  adoption  of  CODP  materials  and  CODP 
therapy  materials  use  was  significantly  associated  with  higher  scores 
on  several  scales  measuring  facilitators  of  adoption  (e.g.,  beliefs  that 
the  CODP  training  was  of  high  quality  and  useful,  feeling  that  col¬ 
leagues  and  supervisors  are  supportive  and  committed  to  the  use  of 
the  CODP).  A  similar  pattern  was  observed  for  CODP  screening  use. 
Higher  overall  use  of  the  CODP  materials  and  use  of  CODP  therapy 
materials  were  also  significantly  associated  with  lower  levels  of  barri¬ 
ers.  Use  of  screening  materials  was  not  significantly  associated  with 
barriers.  Only  one  significant  correlation  was  observed  between  atti¬ 
tudes  variables  (EBPs,  patients  with  mental  health  and  substance  use 
disorders)  and  use  of  CODP  materials.  Specifically,  those  who  used 
the  CODP  more  frequently  overall  also  expressed  higher  openness  to 
using  EBPs. 

Increased  Supervision  Facilitated  Implementation  of  the  CODP 

Over  one-half  (55  percent,  n  =  20)  of  survey  respondents  reported  that 
they  had  received  clinical  supervision  to  help  them  implement  the 
CODP  after  the  training.  They  reported  receiving  a  variety  of  modali¬ 
ties  of  supervision:  in-person  or  telephone  meetings  with  supervisor 
{N  =  13),  peer  consultation  {N  =  11),  live  observation  {N  =  10),  super¬ 
visor  watching  video-recorded  sessions  {N  =  5),  and  group  training 
{N  =  1).  No  respondents  reported  that  a  supervisor  listened  to  audio- 
recorded  sessions.  Nine  respondents  said  they  had  received  over  20 
supervision  sessions  since  the  training;  four  reported  receiving  six  to 
20  supervision  sessions;  and  seven  respondents  said  they  had  received 
five  or  fewer  supervision  sessions.  The  number  of  other  trainings  on 
CODs  ranged  from  zero  to  12  (M  =  2.1,  SD  =  2.5).  A  greater  number 
of  supervision  sessions  was  significantly  correlated  with  more-frequent 
use  of  the  CODP  materials  overall  (r  =  0.70,  p  <  0.01)  and  the  CODP 
therapy  materials  (r  =  0.59,  p  <  0.01),  but  not  with  CODP  screening 
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Correlates  of  Adoption  of  the  CODP  Materials  (A/ : 
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**  p  <  0.01 
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materials.  Interestingly,  adoption  of  the  CODP  materials  overall  or  the 
therapy  materials  was  not  significantly  associated  with  whether  super¬ 
vision  was  received,  hut  use  of  the  CODP  screening  materials  was  sig¬ 
nificantly  correlated  (r  =  0.46,  p  <  0.01).  Participation  in  additional 
trainings  on  treatment  of  CODs  was  not  significantly  correlated  with 
use  of  these  materials. 

Organizational  Characteristics  Were  an  Important  Factor  in 
Successful  Implementation 

In  the  interviews  with  SARP  staff,  the  most  commonly  noted  facilitators 
and  harriers  were  related  to  the  level  of  support  for  the  program  from 
the  chain  of  command  and  fellow  staff.  For  example,  several  interview¬ 
ees  noted  that  having  their  staff  “on  the  same  page”  and  trained  in  the 
program  had  facilitated  implementation.  Conversely,  other  interview¬ 
ees  who  experienced  a  “lack  of  huy-in”  from  the  chain  of  command 
and  other  staff,  or  where  the  chain  of  command  and  other  staff  had  not 
received  training,  found  this  to  he  a  major  harrier  to  implementation. 
Other  organizational  harriers  included  inadequate  resources  (e.g.,  time, 
funding,  supplies)  and  difficulty  collaborating  with  the  mental  health 
department. 

The  Hazelden  Program  Manager  echoed  statements  made  by 
SARP  staff,  highlighting  “synergy  with  the  mental  health  component 
of  their  [SARP]  treatment  services”  and  support  from  SARP  directors 
and  supervisors  as  key  factors  that  could  help  or  hinder  implementa¬ 
tion.  She  said  that  when  SARPs  had  established  working  relationships 
with  mental  health,  implementation  was  more  likely  to  be  successful. 
On  the  other  hand,  SARPs  without  strong  relationships  with  mental 
health,  or  where  mental  health  resources  may  not  even  be  readily  avail¬ 
able,  had  more  difficulty  implementing  the  CODP.  The  SARP  Director 
said  he  had  not  observed  that  SARP  relationships  with  mental  health 
play  a  key  role  in  implementation. 

The  Hazelden  Program  Manager  also  noted  that  inadequate  sup¬ 
port  from  SARP  directors  and  a  lack  of  clinical  supervision  served  as 
barriers  to  implementation.  She  reported  that  when  SARP  directors 
were  not  in  support  of  the  program,  counselors  felt  as  though  they  did 
not  have  to  use  the  CODP.  The  SARP  Director  also  articulated  this 
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view  in  his  interview.  On  the  other  hand,  when  SARP  directors  had 
been  trained  in  the  CODP  and  were  supportive  of  the  program,  both 
the  SARP  Director  and  the  Hazelden  Program  Manager  reported, 
implementation  occurred  much  more  easily.  The  SARP  Director 
observed  that  resistance  from  SARP  staff  similarly  hindered  implemen¬ 
tation.  He  said  he  believed  sending  more  staff  to  the  trainings  would 
aid  implementation. 

Most  Trainees  Reported  the  CODP  Materials  Facilitated 
Implementation,  but  Some  Reported  Needing  Additional 
Implementation  Support 

Some  interviewees  also  reported  that  the  lack  of  specihc  training  on 
how  to  implement  the  program  at  their  particular  site  with  its  patient 
population  made  it  difhcult  to  use  the  CODP.  On  the  other  hand, 
most  interviewees  reported  that  the  design  of  the  CODP  made  it  easy 
to  incorporate  the  training  into  existing  SARP  procedures.  Many 
added  that  the  training’s  instruction  on  the  CBT  modules  and  about 
CODs  in  general  were  particularly  helpful  for  knowing  how  to  effec¬ 
tively  implement  the  CODP. 


Barriers  to  SARP  Implementation  of  CODP  Training 

Barriers  to  Implementation  Were  Not  Frequently  Endorsed  by 
Survey  Respondents 

On  average,  survey  respondents  did  not  report  barriers  to  using  the 
materials,  as  illustrated  by  average  scores  indicating  disagreement  with 
statements  about  barriers  (e.g.,  not  using  the  materials  because  of  a  lack 
of  time,  not  feeling  well  trained,  disagreeing  with  the  philosophy  of  the 
CODP;  Workshop  Assessment  Barriers  Encountered  score:  M  =  21.5 
[possible  scale  range  10-50],  SD  =  7.1). 

Respondents  were  “moderately”  willing  to  adopt  new  EBPs  (total 
EBPs  attitudes  scale:  M  =  1.9,  SD  =  0.3,  with  response  options  rang¬ 
ing  from  0  for  “not  at  all”  to  4  for  “to  a  very  great  extent”).  Subscale 
scores  further  suggest  that  respondents  are  not  resistant  to  the  use  of 
EBPs.  Respondents  indicated  they  would  be  moderately  likely  to  adopt 
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a  new  method  for  delivering  treatment  if  required  by  their  supervisor 
or  the  Navy  (Requirements:  M  =  2.0,  SD  =  0.6).  Staff  also  believed 
they  would  be  more  likely  to  use  a  new  training  if  it  appealed  to  them 
(e.g.,  it  makes  sense,  colleagues  are  happy  using  it)  (Appeal:  M  =  3.0, 
SD  =  0.8).  Respondents  were  also  open  to  trying  new  types  of  interven¬ 
tions  (Openness:  M  =  2.8,  SD  =  0.8).  Respondents  expressed  very  little 
resistance  to  using  research-based  or  manualized  treatments  (Diver¬ 
gence:  M  =  0.9,  SD  =  0.6). 

Respondents  did  not  endorse  negative  beliefs  about  clients  with 
co-occurring  mental  health  problems  (Clinician  Attitudes  Question¬ 
naire:  M  =  1.5,  SD  =  0.6,  with  response  options  ranging  from  1  for 
“strongly  disagree”  to  5  for  “strongly  agree”),  and  there  was  little  evi¬ 
dence  for  negative  attitudes  toward  clients  with  SUDs.  SARP  staff 
disagreed  with  negative  moral  judgments  about  clients  with  SUDs 
(e.g.,  addiction  is  associated  with  weak  will)  (Nonmoralism:  M  =  2.0, 
SD  =  0.7,  with  response  options  ranging  from  1  for  “strongly  disagree” 
to  5  for  “strongly  agree”).  This  is  consistent  with  respondents’  disagree¬ 
ment  with  negative  and  general  statements  about  clients  with  SUDs, 
such  as  “heroin  is  so  addicting  that  no  one  can  really  recover  once  s/he 
becomes  an  addict”  (Nonstereotypes:  M  =  1.8,  SD  =  0.6).  Respondents 
also  disagreed  with  permissive  views  of  substance  use  (e.g.,  “daily  use 
of  one  marijuana  cigarette  is  not  necessarily  harmful”)  (Permissive¬ 
ness:  M  =  2.0,  SD  =  0.6).  Overall,  respondents  had  optimistic  views 
of  the  treatability  of  SUDs  (Treatment  Optimism  subscale:  M  =  4.5, 
SD  =  0.5).  They  also  appear  to  believe  certain  aspects  or  types  of  inter¬ 
vention  (e.g.,  group  therapy)  to  be  important  in  treatment  of  addiction 
(Treatment  Intervention:  M  =  4.0,  SD  =  0.5. 

Site  Visits  Suggested  That  Staff  Reluctance,  Lack  of  Trained  Staff, 
Low  Perceived  Need  for  COD  Services,  and  Low  Actual  Need  for 
COD  Services  Were  Key  Barriers 

At  the  time  of  our  visit.  Site  A'  had  a  specific  program  for  clients  with 
CODs,  where  the  CODP  video  and  CODP  CBT  materials  were  rou- 


'  Site  A:  Self-rated  low  CODP  adopters,  no  web  survey  DDCAT;  2,205  clients  screened  in 
2011. 
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tinely  implemented.  In  the  COD  program,  clients  watch  the  intro¬ 
ductory  video  in  week  1,  and  seven  hour-long  CBT  sessions  from  the 
CODP  are  conducted  over  the  course  of  the  next  three  weeks.  The 
intervention  is  delivered  in  a  group  setting  hy  a  psychologist.  However, 
staff  at  this  site  indicated  that  outside  of  these  CBT  groups,  the  CODP 
is  not  consistently  implemented  in  treatment  for  clients  with  CODs. 
The  site  identified  two  primary  harriers  to  CODP  implementation:  (1) 
lack  of  trained  staff  and  (2)  staff  reluctance.  Site  A  SARP  staff  said  that 
due  to  the  stipulations  of  the  Navy’s  contract  with  Hazelden,  many  of 
their  civilian  contracted  counselors  were  unahle  to  attend  the  CODP 
training.  The  site  was  in  the  process  of  scheduling  on-site  CODP  train¬ 
ings  that  would  he  offered  to  all  SARP  staff  in  order  to  address  this. 
The  second  major  harrier  noted  hy  SARP  staff  was  the  reluctance  of 
some  counselors  to  implement  the  CODP  or  other  treatments  targeting 
CODs.  Site  A  staff  observed  that  SARP  providers  with  formal  mental 
health  training,  such  as  psychologists,  were  generally  implementing  the 
CODP  in  their  practice,  hut  that  the  majority  of  staff,  who  do  not  have 
such  training,  resisted  using  the  program.  SARP  staff  and  adminis¬ 
trators  noted  that  although  many  substance  abuse  counselors  receive 
some  consultation  from  psychologists  who  encourage  the  implemen¬ 
tation  of  the  CODP  for  clients  with  CODs,  most  SARP  counselors 
seem  to  prefer  using  other  techniques  to  focus  primarily  on  SUDs. 
This  suggests  that  staff  reluctance  could  be  related  to  lack  of  training, 
concerns  about  practicing  outside  a  scope  of  practice  that  has  tradition¬ 
ally  focused  only  on  substance  use  issues,  or  not  seeing  the  need  for 
an  alternative  treatment  approach.  SARP  administrators  indicated  that 
hiring  more  staff  with  expertise  in  evidence-based  treatment  of  CODs, 
training  all  SARP  staff  in  the  CODP,  and  offering  CODP  supervision 
following  the  training  could  address  these  barriers  and  improve  imple¬ 
mentation  at  the  site. 

Site  C^,  much  smaller  than  Site  A,  did  not  have  a  specific  pro¬ 
gram  for  clients  with  CODs  at  the  time  of  our  visit.  SARP  adminis¬ 
trators  at  Site  C  estimated  that  approximately  20  percent  of  their  cli- 


^  Site  C:  Self-rated  low  CODP  adopters.  Addiction  Only  Services;  295  clients  screened  in 
2011. 
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ents  have  CODs.  Counselors  reported  providing  treatment  for  CODs 
on  an  as-needed  basis.  Site  C  staff  indicated  that  in  such  cases,  they 
often  use  materials  from  Hazelden’s  “Living  in  Balance”  (LIB)  cur¬ 
riculum  rather  than  from  the  CODE  LIB  is  a  group-oriented  pro¬ 
gram  that  focuses  primarily  on  substance  abuse  but  includes  sessions 
related  to  anger,  negative  emotions,  and  stress.  We  learned  that  clients 
also  attend  “lectures”  focused  on  specific  topics,  which  are  sometimes 
related  to  CODs  (e.g.,  “stress  and  emotions”)  or  use  theoretical  frame¬ 
works  consistent  with  the  CODP  (e.g.,  cognitive  behavioral),  but  that 
do  not  specifically  include  CODP  materials  or  content.  Key  barriers 
to  implementing  the  CODP  at  this  site  included  (1)  staff  reluctance 
and  (2)  low  perceived  need  for  COD  treatment.  Similar  to  Site  A,  at 
Site  C  it  appeared  that  although  staff  had  received  CODP  training, 
they  seemed  to  prefer  other  treatment  curricula,  such  as  the  LIB  cur¬ 
riculum.  Group  therapy  was  reported  to  utilize  supportive  counseling, 
twelve-step,  and  LIB  techniques.  While  some  clients  also  received  indi¬ 
vidual  treatment,  where  CODs  might  be  more  commonly  addressed, 
the  CODP  was  not  reported  to  be  implemented  in  these  settings  either. 
This  may  reflect  reluctance  or  ambivalence  on  the  part  of  counselors 
as  well  as  SARP  leadership.  Implementation  of  the  CODP  was  not 
being  monitored  by  SARP  administration  at  this  site.  Staff  also  said 
it  was  relatively  uncommon  for  their  clients  to  present  with  CODs. 
Therefore,  there  may  have  been  limited  opportunity  to  implement  the 
CODP  at  this  site.  However,  as  described  later  (see  Figure  8.1),  there  is 
room  for  improvement  in  the  assessment  of  CODs  at  Site  C. 

SARP  Site  D3  was  the  smallest  we  visited.  Administrators  reported 
seeing  only  approximately  five  service  member  clients  at  one  time,  along 
with  approximately  two  dependents  (e.g.,  spouses  of  service  members). 
We  learned  that  most  clients  (70  percent),  who  do  not  meet  DSM-IV 
criteria  for  substance  abuse  or  dependence,  go  through  a  brief  video- 
based  educational  program  and  are  then  discharged.  Approximately  30 
percent  receive  additional  treatment,  usually  individual  therapy.  SARP 
counselors  reported  implementing  the  CODP  materials  in  individual 


^  Site  D:  Self-rated  high  CODP  adopters,  Dual  Diagnosis  Capable;  61  clients  screened  in 
2011. 
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therapy  sessions  with  clients  with  CODs,  although  their  doctoral-level 
supervisor  had  not  been  trained  in  the  CODP  and  seemed  not  to  have 
heard  of  the  program.  While  there  appeared  to  he  a  great  deal  of  coun¬ 
selor  enthusiasm  for  the  CODP  at  Site  D,  the  primary  harrier  to  its 
implementation  seemed  to  he  the  small  number  of  clients  at  the  site. 


CHAPTER  EIGHT 


SARP  Capability  to  Provide  Integrated  Care  for 
Co-Occurring  Disorders 


In  this  chapter,  we  describe  the  degree  to  which  Navy  SARPs  are  capa¬ 
ble  to  provide  integrated  care  for  CODs.  We  include  results  from  the 
survey  of  SARP  staff  members  who  participated  in  the  CODP  train¬ 
ing  and  the  site  visits.  The  level  of  integrated  care  provided  at  SARPs 
was  assessed  using  the  DDCAT.  The  DDCAT  was  completed  via 
self-report  on  the  survey  for  those  who  identified  themselves  as  serv¬ 
ing  in  an  administrative  role  within  their  SARP  {N  =  16).  Two  of  the 
16  DDCAT  respondents  came  from  the  same  SARP.  Therefore,  the 
DDCAT  findings  from  the  web  survey  represent  15  different  SARPs. 
The  DDCAT  was  also  completed  by  trained  observers  during  site 
visits.  It  is  important  to  first  note  that  the  DDCAT  is  a  measure  of  a 
site’s  capability  to  provide  integrated  treatment  for  clients  with  CODs. 
It  is  not  a  measure  of  the  quality  of  care  provided,  nor  of  the  use  of  the 
CODP  materials  specifically.  However,  it  does  offer  useful  information 
about  whether  the  physical,  organizational,  programmatic,  and  staffing 
infrastructures  necessary  for  effective  implementation  of  the  CODP 
are  in  place. 

As  described  in  Chapter  Three  and  in  Appendix  A,  the  DDCAT  is 
scored  across  subscales  to  provide  an  assessment  of  the  extent  to  which 
sites  are  able  to  provide  COD  treatment.  The  results  of  the  analysis  are 
presented  by  describing  whether  a  site  falls  into  one  of  three  categories 
(McGovern  et  ah,  2010): 

•  Addiction  Only  Services:  Site  focused  on  providing  services  for 
clients  with  SUDs,  but  not  for  those  with  co-occurring  psycho¬ 
logical  health  disorders. 
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•  Dual  Diagnosis  Capable:  Site  is  able  to  provide  services  for  some 
clients  with  CODs  but  has  higher  capacity  for  those  with  SUDs. 

•  Dual  Diagnosis  Enhanced:  Site  can  provide  services  for  any  client 
with  a  COD.  The  site  can  fully  and  equally  address  both  sub¬ 
stance  use  and  psychological  health  disorders. 


Over  One-Half  of  Survey  Respondents  Reported  Their 
SARP  Capable  of  Providing  Addiction  Only  Services 

Over  one-half  (56  percent,  //  =  9  of  16  administrators)  rated  their 
SARP  as  providing  Addiction  Only  Services.  Self-report  ratings  by 
administrators  at  these  sites  indicated  that  their  SARP  is  not  equipped 
to  treat  clients  with  CODs.  Seven  rated  their  SARP  as  Dual  Diag¬ 
nosis  Capable.  No  respondent  rated  his/her  SARP  as  Dual  Diagno¬ 
sis  Enhanced.  Because  it  is  possible  that  self-reported  DDCAT  scores 
are  confounded  by  the  amount  of  DDCAT  training  received  or  with 
exposure  to  and  training  in  the  CODE,  we  explored  whether  DDCAT 
ratings  were  related  to  the  amount  of  CODP  supervision  and  other 
training  in  treatment  of  COD  that  respondents  had  received.  Theoreti¬ 
cally,  individuals  with  more  DDCAT  training,  or  with  more  knowl¬ 
edge  about  CODs  and  integrated  treatment  for  them,  might  rate  their 
SARPs  differently  compared  to  those  with  less  training  in  these  areas. 
However,  we  found  that  DDCAT  ratings  did  not  significantly  differ 
based  on  the  number  of  CODP  supervision  sessions  received  (P(l)  = 
0.6,  ns)  or  in  the  number  of  other  trainings  on  CODs  (P(l)  =  1.8,  ns). 
Self-report  assessments  of  capability  of  providing  care  for  clients  with 
CODs  using  the  DDCAT  have  limitations,  as  self-reports  tend  to  be 
higher  than  objective  observer  scores.  Yet  it  is  notable  that  over  one- 
half  of  respondents  reported  their  SARP  was  capable  of  providing  only 
addiction  services,  and  that  no  respondent  rated  his  or  her  SARP  in  the 
highest  category  (Dual  Diagnosis  Enhanced). 
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Site  Visits  Identified  Variability  in  Capability  to  Provide 
Integrated  Care  for  CODs 

The  overall  DDCAT  scores  from  the  three  sites  are  shown  below  in 
Table  8.1  (please  refer  to  Substance  Abuse  and  Mental  Health  Services 
Administration  [2011]  for  the  full  list  of  DDCAT  items  within  each 
domain).  A  score  of  5  indicates  the  site  is  Dual  Diagnosis  Enhanced,  a 
3  is  Dual  Diagnosis  Capable,  and  a  1  is  Addiction  Only  Services.  Site 
As  DDCAT  score  of  3.26  indicates  that  the  site  was  rated  to  be  capable 
of  providing  dual  diagnosis  services  overall.  The  other  two  sites’  scores 
(2.16  and  2.37)  suggest  they  are  capable  to  provide  Addiction  Only 
Services,  and  capable  to  provide  dual  diagnosis  services  in  some  but  not 
all  areas.  Table  8.1  shows  the  overall  DDCAT  score  and  dual  diagnosis 
capability  for  each  site. 

Figure  8.1  displays  the  three  sites’  DDCAT  scores  on  each  domain, 
which  provide  some  insight  into  relative  strengths  and  weaknesses.  For 
example,  DDCAT  scores  are  highest  on  training  at  the  sites,  indicat¬ 
ing  that  this  is  a  relative  strength,  whereas  dual  diagnosis  treatment  is 
among  the  lowest  scores  at  each  site,  suggesting  it  is  a  relative  weakness. 

Site  A  had  the  highest  DDCAT  scores  among  the  SARPs  we  vis¬ 
ited.  Site  A  scored  in  the  Dual  Diagnosis  Capable  range  on  program 
structure  and  program  milieu  because  the  SARP  has  a  formalized  col¬ 
laboration  with  mental  health  clinics  on  site,  and  it  openly  displays  and 
distributes  materials  related  to  COD  assessment  and  treatment.  The 
SARP  at  Site  A  demonstrated  that  clinicians  expect  to  see  CODs  and 
efforts  are  made  to  prepare  clinicians  to  be  able  to  treat  mental  health 
disorders  along  with  SUDs,  or  to  refer  them  to  partnering  mental 
health  clinics.  Site  A  was  also  Dual  Diagnosis  Capable  in  assessment  of 


Table  8.1 

Dual  Diagnosis  Capability,  by  Site  {N  =  3) 


Site 

Overall  DDCAT  Score 

Overall  Dual  Diagnosis  Capability 

A 

3.26 

Dual  Diagnosis  Capable 

C 

2.16 

Addiction  Only  Services/Dual  Diagnosis  Capable 

D 

2.37 

Addiction  Only  Services/Dual  Diagnosis  Capable 
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Figure  8.1 

Dual  Diagnosis  Capability  Scores,  by  Domain 


NOTE:  1  =  Addiction  Only  Services,  3  =  Dual  Diagnosis  Capable,  5  =  Dual  Diagnosis 
Enhanced. 

RAND  RR435~8.1 


CODs  due  to  evidence  that  staff  routinely  screened  for  mental  health 
symptoms  and  typically  conducted  more  formal  mental  health  assess¬ 
ments  when  indicated.  Psychiatric  diagnoses  were  often  recorded  in 
client  charts,  and  the  program  was  prepared  to  accept  patients  with 
moderately  severe  mental  health  problems.  Their  score  on  treatment 
was  lower,  hut  approached  the  Dual  Diagnosis  Capable  range.  Site  A’s 
SARP  appeared  to  address  substance  use  as  the  primary  disorder  and 
addressed  mental  health  disorders  secondarily.  There  was  variability  in 
the  extent  to  which  clinicians  reported  on  the  progress  of  mental  health 
problems  and  stages  and  motivation  to  change  among  SARP  clients. 
More  consistency  among  clinicians’  reports  would  help  to  improve  this 
site’s  DDCAT  treatment  score.  Another  strength  of  this  program,  how¬ 
ever,  was  the  implementation  of  evidence-based  programs  for  mental 
health  and  substance  use  problems,  such  as  the  CODP,  particularly 
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among  psychologists  at  the  SARP.  There  was  less  consistent  use  of 
evidence-hased  COD  treatment  among  alcohol  and  drug  counselors, 
however,  although  most  counselors  had  attended  the  CODP  training. 
Site  visit  data  from  Site  A  also  suggested  that  the  SARP  made  an  effort 
to  consider  CODs  in  continuity  of  care  planning.  Although  there  was 
not  a  formalized  protocol  to  manage  mental  health  needs  after  depar¬ 
ture  from  the  SARP,  data  indicated  that  most  clinicians  provided  care 
until  the  client  could  begin  receiving  care  elsewhere.  Peer  support 
groups  for  CODs  (e.g.,  “Double  Trouble”)  were  reported  to  be  lacking 
in  the  community  around  Site  A,  and  staff  indicated  it  was  difficult 
to  connect  clients  with  CODs  to  such  groups  because  of  this.  Staffing 
and  training  were  also  strengths  at  Site  A,  where  a  psychiatrist  worked 
full  time  and  on  site  for  the  SARP.  On-site  supervision  from  psycholo¬ 
gists  was  also  available  for  counselors  as  needed,  and  case  reviews  were 
regularly  conducted  and  included  discussion  of  CODs.  The  primary 
weakness  in  staffing  at  Site  A  was  that  less  than  one-quarter  of  staff 
held  a  mental  health  licensure  (doctoral  or  master’s  level)  or  similar 
competency.  However,  most  direct  care  staff  had  at  least  some  basic 
training  in  how  to  recognize  and  screen  for  mental  health  symptoms, 
and  the  majority  had  also  received  training  in  treatment  of  CODs  via 
the  CODP. 

Site  C  had  lower  DDCAT  scores  on  every  dimension  except  train¬ 
ing,  where  most  staff  had  received  CODP  training.  Site  C  scored  lowest 
on  continuity  of  care.  Administrators  at  this  site  indicated  that  because 
their  clients  come  from  and  are  discharged  to  a  variety  of  geographical 
areas  that  may  not  be  close  to  the  SARP,  it  is  difficult  to  coordinate 
treatment  for  CODs  following  treatment  at  the  SARP.  In  addition, 
however,  the  site  did  not  appear  to  focus  on  recovery  issues  for  mental 
health  disorders.  Instead,  most  continuity  of  care  planning  revolved 
around  SUDs.  Clients  at  the  SARP  at  Site  C  had  limited  access  to 
prescribers.  This  SARP  had  no  formal  relationship  with  a  prescriber, 
which  contributed  to  its  low  DDCAT  staffing  score.  Less  than  one- 
quarter  of  staff  had  mental  health  licensure  or  similar  competency,  but 
these  staff  did  provide  on-site  consultation  and  supervision  for  other 
staff  as  needed.  Case  reviews  did  not  regularly  cover  CODs,  and  peer/ 
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alumni  supports  for  CODs  were  not  available.  These  factors  also  con¬ 
tributed  to  the  lower  staffing  score. 

Site  D  had  similar  DDCAT  scores  to  Site  C,  except  on  assess¬ 
ment,  where  Site  D  scored  in  the  Dual  Diagnosis  Capable  range.  Site 
D  sometimes  collected  mental  health  histories  but  routinely  screened 
for  mental  health  disorders  using  standardized  instruments  with  estab¬ 
lished  psychometric  properties.  Clients  who  screened  positive  on  these 
measures  were  sometimes,  but  not  routinely,  assessed  more  formally.  In 
cases  with  positive  screens,  mental  health  diagnoses  were  usually  docu¬ 
mented.  Site  D  admitted  people  with  moderately  severe  mental  health 
symptoms,  as  long  as  they  were  stable,  whereas  Site  C  indicated  it  was 
unable  to  accept  clients  with  more  than  low  severity  and  persistence  of 
mental  health  problems.  Site  D  also  had  a  consulting  psychiatrist  on 
site,  and  clinicians  regularly  met  with  a  licensed,  supervising  psychia¬ 
trist  to  discuss  cases  as  well.  The  involvement  of  these  mental  health 
staff  contributed  to  the  near-Dual  Diagnosis  Capable  score  on  staffing. 

In  the  next  chapter,  we  provide  a  summary  of  findings,  along 
with  recommendations  for  improving  clinician  training  and  treatment 
delivery  for  CODs. 


CHAPTER  NINE 


Summary  and  Recommendations 


This  evaluation  had  two  main  aims:  (1)  to  understand  the  CODP 
training  approach  and  program  goals  and  (2)  to  describe  the  imple¬ 
mentation  of  the  CODP,  including  trainee  perceptions  of  the  training, 
implementation  of  specific  program  elements,  facilitators  and  barriers 
to  successful  implementation  of  the  CODP,  and  programs’  capability 
to  provide  integrated  care.  This  chapter  summarizes  the  key  findings 
from  the  evaluation  and  describes  limitations  that  should  be  considered 
in  interpreting  our  results.  Finally,  we  offer  suggestions  for  improving 
future  CODs  clinician  training  efforts  and  improving  care  for  CODs. 


Summary  of  Findings 

CODP  Training  Approach  and  Program  Goals 

The  Hazelden  Co-Occurring  Disorders  Clinician  Training  Program  is 
an  evidence-based  program  that  uses  an  integrated  approach  to  treat 
clients  with  co-occurring  psychological  health  and  substance  use  disor¬ 
ders.  Between  2008  and  2010,  approximately  226  Navy  clinician  train¬ 
ees  received  CODP  materials  and  attended  an  offsite  training  ranging 
from  five  to  seven  days  at  Hazelden  in  Center  City,  Minnesota.  Key- 
informant  interviews  indicated  that  the  Navy’s  goal  for  the  CODP 
training  was  to  provide  SARP  staff  with  an  evidence-based  approach 
to  delivering  integrated  treatment  for  CODs.  The  Navy  supported  this 
approach  by  sending  SARP  staff  from  a  range  of  military  installations 
to  receive  training  at  Hazelden. 
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Implementation  of  the  CODP 

Based  on  findings  from  the  survey,  as  well  as  from  interviews  with 
trainees  and  key  informants,  trainees  were  generally  positive  about  the 
CODP  training  and  materials.  For  example,  weh  survey  respondents 
reported  the  training  left  them  feeling  confident  about  their  ability  to 
implement  the  CODP  materials  at  their  SARP.  Interviewed  trainees 
agreed  that  the  training  met  or  exceeded  their  personal  expectations. 
Interviewees  also  noted  areas  where  the  training  could  be  improved. 
These  included  a  need  for  refresher  trainings  and  CODP  supervision 
post-training;  insufficient  information  on  how  to  tailor  the  program 
for  specific  patient  populations  (e.g.,  high  combat  exposure,  severe 
mental  illness);  the  need  for  all  staff,  including  civilians,  administra¬ 
tors,  and  leaders  to  receive  some  training  in  the  program  in  order  to 
increase  buy-in  and  support  for  the  CODP;  a  desire  for  more  time 
observing  Hazelden  practitioners  implementing  the  CODP  during  the 
training;  and  information  on  how  to  modify  the  CODP  for  shorter- 
term  treatment. 

Findings  from  the  survey  and  interviews  suggested  that  the 
screening  and  assessment  materials,  as  well  as  portions  of  the  CBT 
and  ICT  curricula,  were  used  most  frequently  following  the  training, 
while  the  administrator’s  guide  and  medication  management  materials 
were  used  least  frequently.  The  administrators  we  spoke  with  suggested 
refresher  trainings  on  how  to  use  the  DDCAT  would  have  helped  them 
to  sustain  use  of  the  measure.  In  general,  participants  indicated  that 
they  generally  did  not  implement  the  entire  CODP  but  instead  selected 
the  pieces  they  felt  were  most  useful  and  tailored  the  materials  for  their 
clients’  needs.  Interviewees  expressed  interest  in  receiving  clear  direc¬ 
tion  and  guidance  from  leadership  as  to  how  broadly  and  thoroughly 
the  CODP  should  be  implemented.  Some  said  they  believed  it  would 
be  helpful  if  implementation  of  the  materials  were  standardized,  to 
the  extent  possible,  across  SARPs.  Thus,  while  most  study  participants 
reported  feeling  satisfied  with  the  CODP  training,  few  were  fully 
implementing  the  CODP  in  their  clinical  practice. 
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Facilitators  and  Barriers  to  Successful  Implementation 
Overall,  data  from  all  sources  indicated  several  factors  helped  them  to 
implement  the  CODP  in  practice.  This  included  the  high  utility  and 
quality  of  the  CODP  materials,  having  sufEcient  staffing,  supervision, 
resources,  and  time  to  implement  the  program,  and  having  SARP  coun¬ 
selors  with  adequate  training.  Participants  noted  that  encouragement 
from  SARP  directors,  supervisors,  and  fellow  staff  to  use  the  materials 
facilitated  implementation  as  well.  Participants  generally  were  willing 
to  adopt  new  EBPs  and  did  not  hold  negative  attitudes  toward  clients 
with  mental  health  or  substance  use  disorders.  However,  these  attitudes 
were  not  found  to  he  related  to  implementation  of  the  CODP,  except 
that  those  with  more  openness  toward  EBPs  used  the  CODP  more 
frequently.  Interviewed  SARP  staff  added  that  support  from  the  chain 
of  command  was  an  important  facilitator  of  implementation,  as  was 
the  CODP’s  instruction  on  the  CBT  modules  and  about  CODs.  They 
noted  that  difficulty  collaborating  with  mental  health  clinics  could  be 
a  barrier  to  implementation.  Data  from  the  site  visits  further  suggested 
that  SARPs  with  a  low  perceived  need  for  the  CODP,  due  to  having  a 
small  number  of  clients  with  CODs  or  in  general,  led  to  a  weaker  effort 
to  implement  the  program. 

The  Level  of  Integrated  Care  Provided  in  the  Programs 
In  general,  most  of  the  SARPs  for  which  we  obtained  DDCAT  rat¬ 
ings  (i.e.,  via  web  survey  or  site  visits)  did  not  appear  to  be  providing 
integrated  care  for  CODs.  Over  one-half  of  survey  respondents  rated 
their  SARP  as  providing  Addiction  Only  Services,  and  data  from  the 
site  visits  were  consistent  with  this  Ending:  Of  the  three  SARPs  we 
visited,  one  was  rated  as  Dual  Diagnosis  Capable,  and  the  other  two 
as  Addiction  Only  Services.  The  facilitators  and  barriers  to  implemen¬ 
tation  described  above  were  also  associated  with  SARPs’  capacity  to 
provide  integrated  care. 
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Limitations 

It  should  be  noted  that  this  evaluation  has  some  limitations  that  should 
be  considered  when  interpreting  the  results.  First,  the  evaluation  was 
initiated  approximately  two  years  after  the  completion  of  the  last  round 
of  clinician  trainings.  While  this  post  hoc  evaluation  still  holds  value, 
the  evaluation  design  could  be  stronger  if  planned  in  advance  of  the 
training  program.  For  example,  we  were  not  able  to  incorporate  pre¬ 
training  and  posttraining  clinician  assessments  with  validated  mea¬ 
sures,  limiting  our  ability  to  assess  change  in  clinician  skill  over  time. 
In  addition,  we  were  unable  to  monitor  clinician  implementation  of 
the  training  materials  immediately  after  attending  the  training.  Future 
DoD  clinician  training  efforts  would  be  strengthened  by  incorporating 
a  rigorous  evaluation  plan  prior  to  initiating  the  training  program.  This 
could  also  incorporate  assessment  of  patient  outcomes  before  and  after 
training.  However,  the  timing  of  this  data  collection  was  also  a  strength 
of  the  study.  Implementation  of  a  new  treatment  program  takes  time 
and  often  requires  organizational  changes  that  can  be  slow  moving. 
By  assessing  implementation  two  or  more  years  following  the  train¬ 
ing,  this  study  provides  information  about  the  broader,  longer-term 
processes  associated  with  the  implementation  of  the  CODP  training. 

Second,  we  note  limitations  from  the  size  and  representativeness 
of  our  clinician  sample.  We  limited  the  survey  and  interview  sample 
to  trainees  with  active  duty,  reserve,  or  retired  military  status.  Due 
to  regulatory  issues,  we  were  not  able  to  incorporate  the  perspectives 
of  civilian  trainees  who  were  not  retired  military  or  spouses  of  mili¬ 
tary  personnel  (approximately  14  percent  of  trainees,  according  to  data 
obtained  from  Hazelden).  In  addition,  nearly  40  percent  of  trainees  did 
not  receive  the  invitation  to  participate  due  to  inaccurate  contact  infor¬ 
mation.  This  highlights  the  challenges  of  conducting  the  evaluation 
several  months  after  the  completion  of  the  training  program.  It  also 
highlights  the  high  rate  of  staff  turnover  among  trainees.  We  obtained 
an  adjusted  response  rate  of  32  percent.  While  this  response  rate  is  not 
unusual  for  a  clinician  survey,  it  should  be  noted  that  this  represents 
only  16  percent  of  all  trainees.  Therefore,  the  resulting  sample  may  be 
biased  toward  clinicians  who  were  using  the  CODP  materials  more 
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frequently  or  who  were  more  engaged  or  motivated  to  treat  CODs. 
These  factors  contributed  to  a  small  sample  size,  limiting  our  ability  to 
conduct  more  sophisticated  multivariate  analyses. 

Third,  the  data  collected  for  this  evaluation  relied  largely  on  cli¬ 
nician  self-report  on  their  perspectives  of  the  training  and  use  of  the 
CODP  materials.  Due  to  social  desirability  factors,  clinicians  may 
be  more  likely  to  report  favorable  attitudes  and  increased  use  of  the 
CODP  materials  that  may  not  accurately  reflect  clinical  practice.  Also, 
clinicians  received  training  in  these  materials  but  may  not  have  reached 
competence  in  using  the  materials.  Varying  levels  of  familiarity  and 
competence  in  using  the  materials  could  have  biased  their  reports.  We 
were  not  able  to  observe  or  record  treatment  sessions  and  conduct  struc¬ 
tured  coding  of  the  degree  to  which  sessions  adhered  to  the  CODP 
treatment  materials.  These  objective  assessments  of  fidelity  and  compe¬ 
tence  would  have  supplemented  the  clinician  self-report  information. 
Nonetheless,  clinician  perspectives  still  provide  valuable  insight  on  the 
utility  of  training  and  subsequent  implementation. 

Finally,  we  note  that  we  conducted  site  visits  at  only  three  Navy 
SARPs.  Individual  SARPs  vary  in  many  ways  (e.g.,  size,  resources,  staff¬ 
ing  mix)  that  could  affect  the  generalizability  of  the  site  visit  results. 
While  we  aimed  to  select  a  diverse  mix  of  SARPs  for  the  site  visits, 
three  SARPs  simply  cannot  represent  the  diversity  in  individual  SARP 
sites.  Still,  these  site  visits  provided  rich,  detailed  information  regard¬ 
ing  what  may  contribute  to  increased  implementation  of  the  CODP 
materials  and  improved  care  for  CODs.  Despite  these  limitations,  we 
believe  the  integration  of  multiple  data  sources,  both  quantitative  and 
qualitative,  provides  key  insights  in  the  potential  value  of  the  CODP 
training  program. 


Recommendations 

In  this  section,  we  draw  on  key  findings  from  the  evaluation  to  pro¬ 
vide  recommendations  in  how  to  improve  future  efforts  to  train  clini¬ 
cians  in  treating  CODs.  While  this  evaluation  focused  on  the  CODP, 
our  recommendations  are  intended  to  apply  to  the  treatment  of  CODs 
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within  the  military  health  system  more  generally.  It  was  outside  the 
scope  of  the  evaluation  to  make  recommendations  about  the  future  of 
the  CODP,  as  the  evaluation  focused  on  implementation  of  the  pro¬ 
gram  rather  than  its  effectiveness.  Rather,  these  recommendations  can 
he  used  to  inform  the  development  of  and  future  investments  in  pro¬ 
grams  like  the  CODP  that  aim  to  improve  the  delivery  of  integrated 
care  for  service  members  with  CODs. 

Recommendations  to  Improve  Implementation  of  Clinician  Training 
Programs 

Investing  in  clinician  training  in  evidence-based  screening  and  treat¬ 
ment  approaches  demonstrates  a  commitment  to  improving  care  for 
service  members.  Given  the  significant  investment  involved  (e.g.,  train¬ 
ing  funds,  clinician  leave  time),  it  is  essential  that  training  efforts  dem¬ 
onstrate  a  return  on  this  investment.  Research  suggests  that  program 
implementation  is  most  successful  when  there  is  sustained  support  for 
the  program  at  multiple  levels  (e.g.,  SARPs,  installations,  BUMED,  the 
Navy)  and  in  multiple  domains  (e.g.,  financial,  policies,  treatment  and 
training  materials,  staffing,  supervision)  (Bachman  and  Duckworth, 
2003;  Fixsen  et  ah,  2005;  Klein  and  Sorra,  1996;  Powell  et  ah,  2012; 
Rodgers,  Hunter,  and  Rogers,  1993;  Rogers,  1995;  Simpson,  2002). 
We  highlight  how  future  training  efforts  could  be  improved  at  differ¬ 
ent  levels  and  areas  over  time.  Our  recommendations  cut  across  sev¬ 
eral  phases  of  the  implementation  process,  ranging  from  selecting  and 
planning  for  training,  to  the  training  delivery,  and  supporting  imple¬ 
mentation  of  training  materials  following  the  training. 

Recommendation  1.  Develop  a  training  plan  that  specifies 
how  the  training  will  be  implemented  and  sustained,  obtain  lead¬ 
ership  support,  plan  for  staff  turnover,  provide  consultation  after 
training,  and  evaluate  training  efforts. 

Recommendation  la.  Select  a  clinician  training  program  that  pro¬ 
vides  ongoing  consultation  and  implementation  support. 

When  selecting  a  clinician  training,  it  is  essential  to  ensure  the 
content  is  appropriate  and  relevant  for  the  targeted  clinicians.  Training 
programs  that  provide  ongoing  consultation  and  implementation  sup¬ 
port  are  more  desirable,  as  it  is  not  unusual  for  clinicians  to  attend  a 
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didactic  workshop  training  and  not  reach  competence  in  delivering  an 
intervention  (Beidas  and  Kendall,  2010;  Miller  et  ah,  2004;  Sholoms- 
kas  et  ah,  2005). 

Recommendation  lb.  Obtain  organizational  support  from  leadership 
prior  to  initiating  training 

Developing  a  plan  prior  to  initiating  the  training  program  could 
also  increase  the  impact  of  the  training.  This  planning  should  include 
obtaining  huy-in  from  leadership,  both  at  the  clinic  and  base  command 
levels.  SARP  administrators  were  invited  to  attend  the  training,  which 
can  facilitate  engagement  and  support  of  the  treatment.  Yet  attendance 
was  optional,  so  not  all  SARP  administrators  attended.  Administrators 
may  benefit  more  from  a  training  designed  specifically  for  their  needs. 
Further,  providing  options  for  training  mode  (e.g.,  online  versus  in 
person)  may  increase  participation.  An  administrator  training  could  be 
shorter  and  could  focus  on  providing  an  overview  of  the  treatment  pro¬ 
gram  and  its  benefits,  and  how  to  ensure  quality  implementation  of  the 
program  rather  than  specific  techniques  to  deliver  the  treatment.  Then 
SARP  administrators  would  be  trained  to  serve  as  “local  leaders”  who 
could  identify  an  implementation  plan  that  is  appropriate  for  their  site 
(e.g.,  which  components  to  implement,  which  staff  implements  each 
component),  encourage  use  of  materials,  and  problem  solve  as  neces¬ 
sary.  They  can  also  assist  in  ongoing  monitoring  of  the  implementation. 

Recommendation  Ic.  Develop  a  consistent  approach  to  training  rel¬ 
evant  staff  and  strategies  to  address  future  staff  turnover. 

The  plan  should  also  address  which  staff  should  be  trained  and 
include  an  active  plan  to  address  staff  turnover  and  reassignments.  For 
the  CODP  training,  contractors  were  not  trained  because  they  could 
not  travel  to  the  training.  On-site  or  online  training  methods  would 
help  to  ensure  that  contractors,  who  often  stay  at  a  particular  site  for 
longer  periods  than  active  duty  clinicians,  receive  training  as  well.  Staff 
turnover  is  routine;  therefore,  clinician  training  must  be  responsive  to 
this.  Lengthy  offsite  training,  which  tends  to  be  higher-cost  due  to 
travel  and  time  away,  may  not  be  the  most  appropriate  approach  for 
staff  who  may  be  reassigned  to  a  new  position  in  three  years.  Navy 
SARPs  may  want  to  consider  using  a  train-the-trainer  model  (Martino 
et  ah,  2011)  in  which  one  or  two  SARP  staff  are  selected  and  trained 
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as  experts,  who  are  then  available  to  provide  training  to  other  staff  at 
their  site.  Navy  SARPs  may  want  to  select  civilian  staff  (contractors  or 
government  employees)  as  the  first  trained,  as  they  may  have  a  longer 
tenure  at  a  particular  site  than  active  duty  counselors.  This  may  help  to 
increase  the  long-term  value  of  the  training  investment.  If  contractor 
or  civilian  government  employee  staff  members  are  not  able  to  travel  to 
receive  training,  it  is  important  to  identify  other  approaches  to  training 
(e.g.,  online  training,  remote  supervision). 

Recommendation  Id.  Provide  consultation  following  the  training. 

The  likelihood  of  successful  implementation  would  be  increased 
if  trainees  received  ongoing  consultation  or  supervision  to  reach  com¬ 
petence.  While  a  didactic  training  may  be  enough  to  support  integrat¬ 
ing  validated  screening  tools,  a  didactic  training  is  not  enough  to  reach 
competence  in  delivering  an  evidence-based  psychotherapy  (Sholoms- 
kas  et  ah,  2005).  Consultation  or  supervision  following  didactic  train¬ 
ing  is  required  to  reach  competence  and  should  be  integrated  into  any 
future  training  effort. 

Recommendation  le.  Develop  an  evaluation  plan  to  support  assessing 
the  success  of  the  training. 

Finally,  as  mentioned  earlier,  it  is  essential  to  develop  a  plan  in 
advance  to  evaluate  the  training  effort. 

Recommendation  2.  Ensure  clinician  trainings  focus  on  mate¬ 
rials  and  skills  that  are  most  relevant  to  the  site’s  clinical  practice. 

This  evaluation  suggested  that  the  training  provided  by  Hazelden 
was  well  received.  Future  trainings  could  be  improved  by  focusing  on 
particular  components  of  the  materials  that  have  the  best  evidence  base 
and  are  most  relevant  to  the  site’s  clinical  practice.  The  CODP  pro¬ 
gram  materials  are  comprehensive,  yet  it  was  clear  that  some  materi¬ 
als  were  used  frequently  (e.g..  Screening  and  Assessment,  Cognitive 
Behavioral  Therapy,  Integrating  Combined  Therapies)  and  some  mate¬ 
rials  were  used  rarely  (e.g..  Medication  Management,  Family  Program). 
For  example,  SARP  staffers  rarely  integrate  family  members  into  treat¬ 
ment,  so  the  Family  Program  component  was  not  as  relevant  to  their 
practice.  Given  staff  turnover  and  challenges  in  reaching  competence 
in  new  clinical  skills,  less  breadth  and  more  focus  would  increase 
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the  likelihood  that  clinicians  would  he  able  to  use  the  new  materials 
effectively. 

Recommendations  to  Improve  Treatment  for  Co-Occurring  Disorders 
Within  the  Military  Health  System 

The  goal  of  the  CODP  program  was  to  improve  care  for  service  mem¬ 
bers  with  co-occurring  disorders.  Next,  we  provide  some  considerations 
for  how  care  for  COD  delivered  in  the  MHS  could  be  improved.  These 
considerations  incorporate  findings  from  this  evaluation  and  the  litera¬ 
ture  on  improving  care  for  co-occurring  disorders. 

Recommendation  3.  Consider  requiring  that  all  service  mem¬ 
bers  who  receive  care  from  a  SARP  be  screened  for  psychological 
health  problems  using  validated  measures. 

Screening  measures  should  be  used  to  identify  common  co-occur¬ 
ring  psychological  health  conditions  such  as  depression  (e.g.,  PHQ-9 
[Patient  Health  Questionnaire];  Spitzer,  Kroenke,  and  Williams, 
1999),  anxiety  (e.g.,  GAD-7  [Generalized  Anxiety  Disorder  7-item 
scale];  Spitzer  et  ah,  2006),  and  PTSD  (e.g.,  PC-PTSD  [Primary  Care 
Posttraumatic  Stress  Disorder  Screen],  Prins  et  ah,  2003).  Discussions 
with  staff  (by  phone  and  during  site  visits)  suggested  that  SARP  staff 
may  not  routinely  identify  clients  with  CODs.  Routine  screening  of 
service  members  who  enter  formal  treatment,  and  those  who  receive 
less-intensive  interventions  (e.g.,  education  following  a  DUI),  would 
provide  a  mechanism  to  systematically  identify  these  clients.  Routine 
screening  would  also  help  SARP  staff  understand  the  prevalence  of 
CODs  and  need  to  effectively  treat  these  clients. 

Recommendation  4.  Identify  and  certify  select  sites  as  pro¬ 
viding  enhanced  services  for  service  members  with  CODs,  rather 
than  assuming  that  all  SARPs  can  provide  high-quality  care  for 
CODs. 

We  observed  variability  across  SARPs  in  terms  of  their  capa¬ 
bility  to  provide  evidence-based  care  for  CODs.  There  is  still  a  need 
to  improve  the  quality  of  care  to  service  members  with  CODs.  For 
example,  30  percent  of  staff  indicated  their  site  had  a  specialized  COD 
program,  and  one-half  of  respondents  indicated  their  SARP  as  only 
capable  of  providing  addiction  services.  Only  36  percent  of  respon- 
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dents  indicated  they  offered  medication- assisted  therapy,  a  significant 
limitation  given  the  recent  review  confirming  the  effectiveness  of  psy¬ 
chotropic  medications  for  alcohol  (Jonas  et  ah,  2014),  and  only  31  per¬ 
cent  offered  psychiatric  medication,  a  first-line  treatment  option  for 
several  psychiatric  disorders.  Some  sites  may  see  a  very  small  number 
of  clients,  and  it  may  not  he  the  best  use  of  resources  to  provide  special¬ 
ized  treatment  for  CODs  at  every  site.  An  alternative  to  attempting  to 
train  all  staff  at  all  SARPs  to  deliver  COD  care  is  to  identify  particular 
sites  that  have  the  capability  to  provide  evidence-based  care  for  CODs 
and  ensure  that  service  members  with  CODs  get  care  at  those  sites. 
This  may  involve  transferring  a  service  member  to  another  base  to  get 
treatment,  but  we  observed  this  occurring  regularly.  Individual  SARPs 
could  be  reviewed  and  certihed  as  COD  programs  using  a  structured 
site  visit  tool  like  the  DDCAT.  Repeated  on-site  DDCAT  assessments 
by  a  trained,  objective  rater  also  provide  ongoing  feedback  on  spe¬ 
cific  program  changes  that  will  support  improved  COD  care.  Routine 
screening  would  remain  important  to  identify  service  members  who 
might  receive  more  appropriate  treatment  at  a  SARP  certified  in  COD 
care.  Regardless  of  where  a  service  member  receives  treatment  (either 
at  a  nearby  military  treatment  facility  or  at  another  military  treatment 
facility  that  specializes  in  CODs),  all  service  members  should  have 
access  to  a  site  that  is  capable  of  providing  care  for  CODs. 

Recommendation  5.  Implement  measures  to  assess  the  qual¬ 
ity  of  care  provided  at  SARPs,  including  both  process  and  out¬ 
comes  measures. 

We  observed  few  mechanisms  to  monitor  the  quality  of  care 
delivered  to  service  members  for  CODs,  either  at  individual  SARPs 
or  across  all  SARPs.  For  example,  we  did  not  identify  any  measures 
assessing  treatment  process  (e.g.,  use  of  evidence-based  psychotherapy), 
which  limits  the  ability  to  determine  whether  the  care  delivered  is 
consistent  with  clinical  practice  guidelines.  In  addition,  there  are  cur¬ 
rently  no  systematic  approaches  for  ongoing  monitoring  of  client  treat¬ 
ment  outcomes.  Integrating  outcomes  monitoring  provides  important 
information  on  the  effectiveness  of  the  services  delivered.  Further,  out¬ 
comes  monitoring  is  essential  to  delivering  measurement-based  care,  in 
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which  repeated  assessments  of  client  outcomes  help  to  guide  treatment 
delivery. 


Conclusion 

This  report  presents  the  results  of  an  evaluation  of  a  clinician  training 
program  to  improve  care  for  Navy  personnel  with  CODs.  These  results 
suggest  that  clinicians  perceived  a  need  for  such  training,  were  open 
to  receiving  it,  and  generally  viewed  the  training  experience  positively. 
However,  the  results  also  suggest  variability  in  the  level  of  integrated 
care  delivered  hy  the  SARPs  and  indicate  that  many  SARPs  may  not 
he  providing  integrated  care.  Results  indicate  that  more  careful  plan¬ 
ning  and  targeting  of  SARPs  best  suited  to  develop  skills  for  treating 
CODs  would  improve  the  effectiveness  of  the  training  and,  therefore, 
be  more  likely  to  translate  into  higher-quality  care  for  service  members 
with  CODs. 


APPENDIX  A 


Additional  Methodological  Details 


In  this  appendix,  we  provide  more  methodological  detail  about  our 
procedures. 


Web  Survey  Measures 

The  survey  (Appendix  C)  assessed  a  number  of  domains  described 
below.  Alphas  were  calculated  using  the  web  survey  data  collected  for 
this  study. 

Demographics  and  Professional  Characteristics  were  collected, 
including  age,  sex,  race/ethnicity,  provider  type  (e.g.,  alcohol  and  drug 
counselor,  clinical  psychologist),  primary  professional  role  at  the  SARP 
(clinician,  administrator,  both),  current  SARP  location,  and  were  asked 
whether  they  were  currently  at  the  same  site  at  the  time  they  received 
CODP  training. 

Characteristics  of  SARP s  were  assessed  using  selected  items  from 
the  National  Survey  of  Substance  Abuse  Treatment  Services  (Depart¬ 
ment  of  Health  and  Human  Services,  2008).  Items  assess  types  of 
services  available  at  the  SARP,  the  proportion  of  clients  who  receive 
individual  or  group  counseling,  and  how  often  various  therapeutic 
approaches  (e.g.,  CBT)  are  used  at  the  SARP. 

To  mQzsuK  frequency  of  use  of  CODP  materials  since  the  train¬ 
ing,  we  developed  nine  items,  which  were  rated  on  a  five-point  scale 
(0  =  Never,  1  =  Once  or  twice,  2  =  A  few  times  a  year,  3  =  Monthly, 
4  =  Weekly).  Respondents  were  asked,  “Which  of  the  following  best 
characterizes  how  often  you  used  these  CODP  materials  since  the 
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training?  Examples  of  ‘use’  include  using  handouts  or  measures  in 
assessment  or  treatment,  reading  or  reviewing  the  manuals,  or  using 
content  to  guide  a  session.”  Respondents  were  asked  to  rate  the  fre¬ 
quency  with  which  they  have  used  the  CODE  program  curricula  and 
specihc  program  materials,  a  total  of  nine  items  (five  curricula  items: 
Administrator’s  Guidebook,  Screening  and  Assessment,  Integrat¬ 
ing  Combined  Therapies,  Cognitive  Behavioral  Therapy,  Medication 
Management;  and  four  materials  items:  Clinician’s  Guide,  workbook 
factsheets,  workbook  handouts,  screening/assessment  measures).  To 
examine  factors  that  might  be  associated  with  overall  implementation 
of  the  CODE  materials,  we  summed  the  scores  of  all  nine  variables  to 
create  an  “Adoption  of  CODE  Materials”  total  score  (a=0.92).  To  eval¬ 
uate  differential  implementation  of  screening  and  treatment  materials, 
we  computed  indicators  of  (1)  adoption  of  CODE  screening  materi¬ 
als,  and  (2)  adoption  of  CODE  therapy  materials.  The  screening  vari¬ 
able  was  a  sum  of  two  items:  frequency  of  use  of  the  CODE  screening 
and  assessment  guide,  and  frequency  of  use  of  the  CODE  screening 
and  assessment  measures.  The  indicator  of  adoption  of  CODE  therapy 
materials  was  a  sum  of  the  items  assessing  frequency  of  use  of  the  CBT 
and  ICT  guides. 

Implementation  and  perceptions  of  the  CODP  materials,  and  bar¬ 
riers  to  implementation  were  measured  using  the  Texas  Christian 
University  Workshop  Assessment  Follow-Up  Workshop  Assessment 
(Bartholomew  et  ah,  2007).  The  Workshop  Assessment  is  a  26-item 
questionnaire  with  acceptable  reliability  and  validity  that  measures  use 
and  intentions  to  use  training  materials,  program  resources  related  to 
use,  and  barriers  to  use.  We  used  several  subscales  of  this  measure  that 
assessed  the  quality  of  the  training,  the  quality  and  usefulness  of  the 
training  materials,  ability  of  participants  to  incorporate  the  materi¬ 
als  in  their  workplaces,  perceived  support  by  participants’  workplaces 
for  implementing  the  training,  and  barriers  to  implementation.  The 
Training  subscale  (three  items;  a  =  0.76)  assesses  participants’  views  on 
the  quality  of  the  training.  The  Quality  and  Utilization  subscale  (four 
items;  a  =  0.93)  assesses  training  participants’  views  about  the  quality 
and  usefulness  of  the  materials.  The  Resources  and  Skills  subscale  (five 
items;  a  =  0.77)  measures  participants’  ability  to  incorporate  the  mate- 
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rials  in  their  workplaces.  The  Support  and  Commitment  suhscale  (five 
items;  a  =  0.89)  identifies  the  extent  to  which  implementing  the  skills 
learned  at  the  training  has  been  supported  hy  colleagues  and  senior 
staff  and  has  been  sustainable.  Lastly,  the  Barriers  Encountered  sub¬ 
scale  (eight  items;  a  =  0.88)  was  used  to  determine  why  participants  did 
not  use  materials  in  their  practice.  All  items  were  rated  on  a  five-point 
scale  (1  =  Disagree  Strongly,  2  =  Disagree,  3  =  Undecided,  4  =  Agree, 
5  =  Agree  Strongly),  with  one  item  on  the  Resources  and  Skills  subscale 
reverse  coded.  Rescaled  average  scores  were  computed  for  the  total  and 
subscales  (Bartholomew  et  ah,  2007).  To  do  this,  the  average  score  is 
computed  and  then  multiplied  by  10.  Therefore,  scale  scores  can  range 
from  10  to  50. 

The  utility  of  the  CODP  was  assessed  using  an  eight-item  measure 
created  by  Hazelden  to  assess  the  utility  of  the  program.  Psychomet¬ 
ric  properties  are  not  available  for  this  measure.  Questions  assess  self- 
efficacy  and  achievement  of  different  program  objectives.  Items  are 
rated  on  a  four-point  scale,  where  4  =  Strongly  agree,  3  =  Somewhat 
agree,  2  =  Somewhat  disagree,  1  =  Strongly  disagree.  The  first  three 
items  were  asked  only  of  respondents  who  reported  serving  in  adminis¬ 
trative  roles.  Because  the  items  assess  diverse  concepts,  these  items  were 
examined  individually. 

Respondents  also  reported  on  the  supervision  and  additional 
training  to  support  implementation  of  the  CODP  they  received  after  the 
Hazelden  training.  Specifically,  respondents  were  asked  whether  they 
had  ever  received  any  clinical  supervision  to  help  perform  assessments 
or  deliver  treatment  using  the  CODP  materials.  If  they  answered 
“yes,”  they  were  also  asked  to  report  which  modality  of  supervision 
was  received  (e.g.,  live  observation,  face-to-face  meetings  with  supervi¬ 
sor,  supervisor  listened  to  audio  recordings  of  sessions),  and  to  estimate 
the  number  of  CODP  supervision  sessions  they  had  received  since  the 
training  (0  =  None,  1  =  1-5,  2  =  6-10,  3  =  11-15,  4  =  1620,  5  =  21  or 
more).  Finally,  respondents  reported  the  number  of  other  trainings  on 
CODs  they  had  attended  since  the  CODP  training  at  Hazelden. 

Attitudes  about  the  adoption  of  EBPs  were  measured  with  the 
15-item  Evidence-Based  Practice  Attitude  Scale,  which  has  accept¬ 
able  reliability  and  validity  (Aarons,  2004).  This  measure  was  selected 
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because  resistance  to  using  EBPs  could  be  a  barrier  to  implementation 
of  the  CODP.  We  used  a  number  of  subscales  to  assess  trainees’  per¬ 
ceptions  of  using  EBPs,  including  subscales  that  assessed  the  appeal 
of  EBPs,  the  influence  of  regulatory  requirements,  openness  to  new 
interventions,  and  views  on  clinical  research.  For  the  purposes  of  the 
current  study,  the  item  “if  you  received  training  in  a  therapy  or  inter¬ 
vention  that  was  new  to  you,  how  likely  would  you  be  to  adopt  it  if 
it  was  required  by  your  state”  was  removed  from  the  Appeal  subscale 
because  it  is  not  relevant  for  SARP  staff.  Therefore,  there  were  14  evi¬ 
dence-based  attitudes-related  items  administered  in  the  current  study. 
The  four  attitude  subscales  consist  of  Appeal  (four  items;  a  =  0.92), 
Requirements  (two  items;  a  =  0.92),  Openness  (four  items;  a  =  0.91), 
and  Divergence  (four  items;  a  =  0.39).  The  Appeal  subscale  measures 
the  extent  to  which  the  respondent  would  adopt  a  new  practice  if  it 
were  intuitively  appealing,  made  sense,  could  be  used  correctly,  or  is 
being  used  by  colleagues  who  are  happy  with  it.  The  Requirements 
subscale  asks  respondents  whether  they  would  adopt  a  new  practice 
if  it  were  required  by  an  agency,  supervisor,  or  state.  The  Openness 
scale  assesses  the  extent  to  which  the  respondent  is  generally  open  to 
trying  new  interventions  and  would  be  willing  to  try  or  use  new  types 
of  therapy.  Eastly,  the  Divergence  subscale  measures  the  respondent’s 
views  on  the  clinical  utility  of  research-based  interventions  and  relative 
importance  of  clinical  experience  (Aarons,  2004).  In  addition  to  the 
subscale  scores,  a  total  score  was  calculated  (14  items;  a  =  0.92),  reflect¬ 
ing  global  attitude  toward  the  adoption  of  EBPs.  Respondents  were 
asked  to  answer  each  question  based  on  a  0-4  scale  (0  =  Not  at  all,  1 
=  To  a  slight  extent,  2  =  To  a  moderate  extent,  3  =  To  a  great  extent,  4 
=  To  a  very  great  extent).  Mean  scores  were  used  for  analyses  including 
attitudes  total  and  subscales. 

Attitudes  about  treating  clients  with  co-occurring  disorders  were 
assessed  with  the  Clinician  Attitudes  Questionnaire  (Hunter  et  ak, 
2005).  This  measure  consists  of  seven  items  (e.g.,  “there  is  little  to  be 
done  for  clients  that  are  mentally  ill”),  which  are  rated  on  a  five-point 
scale  (1  =  Strongly  disagree,  2  =  Somewhat  disagree,  3  =  Neither  dis¬ 
agree  nor  agree,  4  =  Somewhat  agree,  5  =  Strongly  agree).  A  mean 
attitudes  score  was  computed  (a  =  0.83)  (Hunter  et  ah,  2005).  This 
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measure  was  selected  because  negative  attitudes  about  individuals  with 
mental  health  problems  (e.g.,  “People  with  mental  illness  have  weak 
personalities”)  can  serve  as  barriers  to  delivering  effective  treatment  for 
CODs. 

Attitudes  about  treating  clients  with  SUDs  were  measured  with 
the  Brief  Substance  Abuse  Attitude  Survey  (Chappel,  Veach,  and 
Krug,  1985).  This  is  a  20-item  survey  derived  from  the  longer  Sub¬ 
stance  Abuse  Attitude  Survey.  These  items  are  rated  a  five-point  scale 
(1  =  Strongly  disagree,  2  =  Disagree,  3  =  Uncertain,  4  =  Agree, 
5  =  Strongly  agree).  This  measure  was  selected  because  negative  atti¬ 
tudes  about  clients  with  SUDs  could  be  another  barrier  to  effective 
implementation  of  the  CODP.  This  measure  consists  of  five  subscales, 
which  assess  respondent  beliefs  about  substance  use,  attitudes  about 
substance  abuse  interventions,  common  substance  abuse  stereotypes, 
views  on  the  utility  of  substance  abuse  treatment,  and  moral  issues 
around  substance  abuse.  The  Permissiveness  subscale  (four  items;  a 
=  0.32)  measures  beliefs  about  substance  use.  The  Treatment  Inter¬ 
vention  subscale  (five  items;  a  =  0.44)  assesses  attitudes  toward  vari¬ 
ous  substance  use  interventions.  The  Non-Stereotypes  subscale  (three 
items;  a  =  0.43)  assesses  beliefs  about  common  substance  use  stereo¬ 
types.  The  Treatment  Optimism  subscale  (five  items;  a  =  0.50)  que¬ 
ries  about  beliefs  that  substance  use  treatment  can  lead  to  a  successful 
outcome.  Lastly,  the  Non-moralism  subscale  (three  items;  a  =  0.43) 
assesses  views  on  moral  issues  surrounding  substance  use  and  misuse. 
Hazelden  added  nine  items  to  the  measure  to  specifically  inquire  about 
attitudes  about  CODs  and  related  issues,  but  these  nine  items  were 
not  analyzed  for  the  current  study,  as  they  showed  little  variability  and 
have  not  been  empirically  examined  in  previous  studies. 

Participants  who  were  SARP  administrators  were  asked  to  assess 
the  capability  of  their  SARP  to  provide  services  for  clients  with  co-occur- 
ring disorders  using  the  DDCAT,  version  4.0.  The  DDCAT  is  a  35-item 
measure  that  was  developed  to  assess  capability  to  assess  and  treat  co¬ 
occurring  disorders  and  to  guide  addiction  treatment  providers  in 
enhancing  services  for  persons  with  co-occurring  disorders.  The  reli¬ 
ability  and  validity  of  the  DDCAT  are  acceptable  (McGovern,  Matz- 
kin,  and  Giard,  2007).  The  DDCAT  was  presented  and  described  to 
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CODP  participants  at  the  training.  The  DDCAT  was  designed  to 
he  rated  hy  trained  observers  who  use  information  from  a  variety  of 
sources  (e.g.,  staff  and  client  interviews,  observation  of  clinic  materi¬ 
als  and  procedures,  chart  review)  to  compute  scores  across  the  seven 
domains.  However,  because  it  was  not  possible  for  the  research  staff  to 
visit  every  site,  we  asked  SARP  administrators  to  rate  their  own  SARPs 
on  the  web  survey.  Self-administered  DDCAT  scores  tend  to  be  posi¬ 
tively  biased  (i.e.,  toward  more  desirable  scores)  by  about  one  point  on 
average  (Lee  and  Cameron,  2009).  We  modified  DDCAT  items  to  be 
appropriate  for  self-report  (as  opposed  to  observer  rated)  and  for  the 
SARP  setting.  We  consulted  with  the  primary  author  of  the  DDCAT 
(Mark  McGovern),  who  reviewed  the  modifications. 

The  DDCAT  includes  35  items  that  are  rated  on  a  Likert  Scale. 
The  individual  items  assess  the  extent  to  which  a  site  is  able  to  pro¬ 
vide  COD  treatment.  Example  items  include  “Are  patients  with  mental 
health  disorders  expected  and  welcomed  at  your  SARP?”  and  “To  what 
extent  is  education  about  psychiatric  disorders,  their  treatment,  and 
their  interaction  with  substance  use  &  its  treatment  offered  at  your 
SARP?”  The  ratings  range  from  1  to  5,  where  1  indicates  that  the 
SARP  is  capable  of  providing  little  to  no  treatment  for  mental  health 
conditions,  3  indicates  that  the  site  can  provide  some  COD  treatment, 
and  5  indicates  that  the  site  provides  comprehensive  COD  treatment. 
The  DDCAT  consists  of  seven  subscales,  which  assess  a  SARP’s  pro¬ 
gram  structure,  program  milieu,  assessment  of  CODs,  treatment  of 
CODs,  ability  to  provide  continuity  of  care,  stafhng,  and  training.  We 
computed  mean  subscale  scores  for  each  DDCAT  subscale.  Then  we 
averaged  the  mean  subscale  scores  to  create  a  total  score  for  the  SARP, 
which  can  range  from  1  (Addiction  Only  Services)  to  5  (Dual  Diag¬ 
nosis  Enhanced)  and  indicates  the  site’s  overall  level  of  dual  diagnosis 
capability. 


Site  Visit  Selection 

Survey  DDCAT  ratings  of  Addiction  Only  Services  were  considered 
to  indicate  low  levels  of  integrated  treatment,  whereas  Dual  Diagnosis 
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Capable  ratings  were  considered  to  indicate  higher  levels  of  integrated 
treatment.  To  categorize  individuals  into  “high”  or  “low”  adopters  of 
the  CODP  materials,  we  counted  the  number  of  “adoption”  variables 
on  which  each  respondent’s  rating  fell  either  one  standard  deviation 
below  (low  adoption)  or  above  the  sample  mean  (high  adoption).  The 
possible  range  for  these  variables  was  0  to  9.  For  example,  a  respon¬ 
dent’s  frequency  of  use  might  be  one  standard  deviation  below  the 
mean  for  two  variables,  one  standard  deviation  above  the  mean  for 
three  variables,  and  within  one  standard  deviation  of  the  mean  on  the 
remaining  four  variables.  Yet,  another  respondent  could  have  reported 
using  all  nine  CODP  materials  at  a  frequency  one  standard  deviation 
above  the  mean.  This  respondent  would  be  considered  a  high  adopter. 
Next,  we  grouped  respondents  by  SARP  and  simultaneously  exam¬ 
ined  respondents’  ratings  on  the  DDCAT  and  the  number  of  materials 
respondents  reported  using  at  a  frequency  falling  one  standard  devia¬ 
tion  below  or  above  the  mean.  Given  our  reliance  on  self-reported  data 
to  select  sites,  we  aimed  to  select  sites  where  responses  reliably  reported 
high  or  low  implementation  of  the  CODP  and  integration  of  COD 
treatment.  Therefore,  in  cases  where  we  had  multiple  respondents  from 
the  same  SARP,  we  selected  those  where  respondents’  reports  were  con¬ 
sistent  in  level  of  dual  diagnosis  capability  (from  the  DDCAT)  and/or 
level  of  adoption  of  the  CODP  materials.  If  there  was  only  one  respon¬ 
dent  from  a  SARP,  we  selected  sites  where  that  respondent’s  DDCAT 
and  adoption  ratings  were  consistent  with  one  another  (e.g.,  rating 
Dual  Diagnosis  Capable  on  the  DDCAT  and  reporting  high  adoption 
of  CODP  materials).  Of  the  19  SARPs  for  which  we  had  DDCAT  and/ 
or  adoption  data,  we  removed  seven  sites  from  consideration  due  to 
inconsistent  ratings.  Another  SARP  was  removed  from  the  list  because 
its  overseas  location  made  a  site  visit  unfeasible.  We  then  considered  the 
size  of  the  remaining  1 1  sites  using  data  from  BUMED  on  number  of 
clients  screened  and  treated,  as  well  as  resources  available  to  each  site 
(e.g.,  co-location  with  a  major  medical  center). 

We  selected  two  “large,  high-resource”  sites,  and  two  “small,  low- 
resource”  sites,  weighting  SARPs  for  which  web  survey  respondents 
had  completed  the  DDCAT  {N  =  6  of  remaining  11)  more  heavily. 
Within  each  size  category,  there  was  one  “high  implementer/adopter” 
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and  one  “low  implementer/adopter.”  It  is  important  to  note  that  select¬ 
ing  a  site  as  a  high  or  low  implementer  of  the  CODP  does  not  necessar¬ 
ily  reflect  its  capability  to  treat  CODs.  Also,  self-reported  information 
on  level  of  implementation  was  available  only  from  staff  members  who 
responded  to  the  web  survey.  Other  staff  members  at  each  SARP  may 
have  a  higher  or  lower  level  of  implementation  of  the  CODP  materials. 


APPENDIX  B 


Key- Inform  ant  Discussion  Guide 


Key-Informant  Questions — Hazelden 

Hazelden  Goals/Expectations 
Questions  to  Initiate  Discussion 

1.  What  were  your  goals  and  expectations  for  implementation  of 
the  CODP  into  the  SARPs? 

Possible  Prompts/Probes 

2.  Do  you  think  your  goal  of _ was  met? 

3.  If  you  delivered  the  program  to  SARP  staff  again,  what  would 
you  do  differently? 


Perceived  Strengths  for  SARPs 
Questions  to  Initiate  Discussion 

1.  In  what  ways  do  you  think  the  CODP  has  impacted  the  SARPs? 

Possible  Prompts/Probes 

2.  What  were  the  specific  strengths  of  the  program? 

3.  What  were  the  program’s  limitations? 
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Barriers/Facilitators  of  Implementation 
Questions  to  Initiate  Discussion 

1.  What  factors  influenced  how  easy  or  difficult  it  was  for  the 
SARPs  to  implement  the  CODP? 

Possibie  Prompts/Probes 

2.  What  supports  did  you  plan  to  he  there  to  help  people  imple¬ 
ment  skills?  What  supports  were  actually  there? 

3.  What  got  in  the  way  of  effective  implementation? 

4.  What  made  it  easier  to  implement  the  CODP? 


Key  Informant  Questions — BUMED 

Goals  and  Expectations  for  Program 
Questions  to  Initiate  Discussion 

1.  What  were  your  goals  and  expectations  for  selecting  the  CODP 
for  implementation  in  the  SARPs? 

Possibie  Prompts/Probes 

2.  How  did  you  select  the  CODP  program  for  implementation  in 
SARPs? 

3.  Do  you  think  the  program  met  the  Navy’s  goals  of _ ? 


Perceived  Strengths  for  SARPs 
Questions  to  Initiate  Discussion 

1.  In  what  ways  do  you  think  the  CODP  has  impacted  the  SARPs? 

Possibie  Prompts/Probes 

2.  What  were  the  specific  strengths  of  the  program? 

3.  What  were  the  program’s  limitations? 
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Barriers/Facilitators  of  Implementation 
Questions  to  Initiate  Discussion 

1.  What  factors  influenced  how  easy  or  difficult  it  was  for  the 
SARPs  to  implement  the  CODP? 

Possible  Prompts/Probes 

2.  What  supports  did  you  plan  to  he  there  to  help  people  imple¬ 
ment  skills?  What  supports  were  actually  there? 

3.  What  got  in  the  way  of  effective  implementation? 

4.  What  made  it  easier  to  implement  the  CODP? 


BUMED  Perception  of  Program  Value 
Questions  to  Initiate  Discussion 

1.  Tell  me  about  the  value  of  the  program  to  the  Navy. 

Possible  Prompts/Probes 

2.  What  did  the  program  cost? 

3.  Do  you  think  the  benefits  of  the  program  were  worth  the  costs? 

4.  What  factors  will  influence  your  decision  to  renew  the  CODP 
contract? 

5.  Are  you  planning  to  do  this  type  of  training  again?  Why  or  why 
not? 

6.  Would  you  work  with  Hazelden  again  for  this  type  of  training? 
Why  or  why  not? 


APPENDIX  C 

Web  Survey 


Table  C.1 

Online  Program  Survey 


Question 


Response  Option 


Which  of  the  following  Active  Duty  or  in  an  Active  status; 
best  describes  your  current  Retired  from  the  US  Military; 
status?  Reserve  status  or  National  Guard; 

Spouse  of  a  service  member; 

Civilian 

If  option  5:  Unfortunateiy,  you  are 
not  eiigible  to  compiete  this  survey 
as  per  our  current  institutional 
review  board  approvals.  Thank  you 
for  your  time. 

Demographics 


Sex 

Are  you 

0. 

1. 

Male 

Female 

Age 

How  old  are  you? 

Range:  18-99 

Ethnicity 

Are  you  of  Hispanic  or 

0. 

No 

Latino  origin  or  descent? 

1. 

Yes 

Race 

What  is  your  race?  Please 

1. 

White 

select  one  or  more. 

2. 

Black  or  African  American 

3.  Asian 

4.  Native  Hawaiian  or  Other 
Pacific  Islander 

5.  American  Indian  or  Alaskan 
Native 

6.  NONE  OF  THESE  (Please 

describe _ ) 


SCREENING  FOR 
SERVICE 

MEMBERS  ONLY 
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Table  C.1 — Continued 


Question 


Response  Option 


We  would  like  to  compare 
the  results  of  this  survey 
to  results  from  similar 
measures  that  were 
completed  at  the  Co- 
Occurring  Disorders 
Program  (CODP)  Hazelden 
training.  To  do  this,  we 
will  need  to  know  the  ID 
number  you  created  and 
used  at  the  training.  The 
following  three  questions 
will  tell  us  what  your  ID 
number  was. 

What  is  the  first  letter  Fill  in  (A-Z) 

of  your  mother's  maiden 

name? 

What  is  the  first  digit  Fill  in  (0-9) 

of  your  social  security 

number? 


What  is  the  last  digit  Fill  in  (0-9) 

of  your  social  security 

number? 


Administrative 


Current  SARP  At  which  site  are  you 
currently  located? 


1.  29  Palms 

2.  77  Bush 

3.  Bahrain 

4.  Bethesda 

5.  Bremerton 

6.  Brunswick 

7.  Charleston 

8.  Corpus  Christ! 

9.  Diego  Garcia 

10.  Fallon 

11.  Great  Lakes 

12.  Groton 

13.  Guantanamo  Bay 

14.  Guam 

15.  Gulfport 

16.  Jacksonville 

17.  Key  West 

18.  Kings  Bay 

19.  Lejeune 

20.  LeMoore 
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Table  C.1 — Continued 


Question 


Response  Option 


21.  Naples 

22.  Naval  Health  Clinic  Annapolis 

23.  Naval  Health  Clinic  New 
EnglancJ 

24.  Naval  Medical  Center 
Portsmouth 

25.  Navy  Drug  and  Alcohol  Coun¬ 
selor  School 

26  New  London 

27.  Newport,  Rl 

28.  Norfolk 

29.  Oak  Harbor 

30.  Okinawa 

31.  Parris  Island 

32.  Pax  River 

33.  Pearl  Harbor 

34.  Pendleton 

35.  Port  Hueneme 

36.  Port  Loma 

37.  Rota,  Spain 

38.  San  Diego 
39  Sasebo 

40.  Sigonella 

41.  Stennis 

42.  USS  Abraham  Lincoln 

43.  USS  Frank  Cable 

44.  USS  George  Washington 

45.  USS  Harry  S.  Truman 

46.  USS  Nimitz 

47.  USS  Ronald  Reagan 

48.  Whidbey  Island 

49.  Willow  Grove 

50.  Yokosuka 

51.  Other  (PLEASE  SPECIFY: 
_ ) 


Changed  SARP  When  you  attended  the  0.  No 

since  training  Co-Occurring  Disorders  1.  Yes 

Program  (CODP)  Hazelden 
training,  were  you  located 
at  this  same  site? 
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Table  C.1 — Continued 


Question 


Response  Option 


Provider  type  What  type  of  provider  are 
you? 


1.  Certified  Drug  Abuse  Coun¬ 
selor  (CADAC) 

2.  Clinical  Psychologist 

3.  Clinical  Social  Worker 

4.  Licensed  Chemical  Depen¬ 
dency  Counselor  (LCDC) 

5.  Navy  Alcohol  and  Drug  Coun¬ 
selor  I  (ADCI) 

6.  Navy  Alcohol  and  Drug  Coun¬ 
selor  II  (ADCII) 

7.  Licensed  Independent 
Practitioner 

8.  Psychiatric  Nurse 

9.  Psychiatrist 

10.  NONE  OF  THESE  (PLEASE 
DESCRIBE:  [FILL  IN  BLANK]) 


Position 


What  is  your  primary 
professional  role  at  your 
SARP? 


1.  Administrator 

2.  Counselor  or  Clinician 

3.  Administrator  and  Counselor/ 
Clinician 


Supervision  The  following  questions 

ask  about  the  amount 
and  type(s)  of  supervision 
you  received  after  the 
Co-Occurring  Disorders 
Program  (CODP)  training 
at  Hazelden  to  help  you 
implement  the  skills  you 
learned.  "Supervision" 
is  feedback  or  coaching 
delivered  to  the  treatment 
provider  by  another 
professional  with  relevant 
experience  and/or 
expertise. 


Supervision  After  you  completed  the  0.  No 

ever  CODP  training  at  Hazelden,  1.  Yes 

did  you  receive  any  clinical 
supervision  to  help  you 
perform  assessments  and/or 
deliver  treatment  using  the 
CODP  materials? 
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Table  C.1 — Continued 


Question 

Response  Option 

Supervision 

What  modality  of  CODP 

1. 

Face  to  face  (in  person)  or 

type 

supervision  do/did  you 

telephone  meetings  with 

receive?  (Check  all  that 

supervisor 

apply.) 

2. 

Supervisor  listened  to  audi 
recorded  sessions 

3. 

Supervisor  watched  video 
recorded  sessions 

4. 

Live  observation 

5. 

Peer  consultation 

6. 

Other  (PLEASE  DESCRIBE: 
[FILL  IN  BLANK]) 

Number 

Please  estimate  the  number 

1. 

None 

supervision 

of  CODP  supervision 

2. 

1-5 

sessions 

sessions  you  have  had  since 

3. 

6-10 

the  training. 

4. 

11-15 

5. 

16-20 

6. 

21  or  more 

other 

How  many  other  trainings 

0- 

-100 

trainings  on 

on  co-occurring  disorders 

COD  since 

have  you  participated  in 

CODP 

since  you  completed  the 
CODP  program? 
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Table  C.1 — Continued 


Question 


Response  Option 


Facility 

characteristics 


Types  of 
services 


Please  consider  the  primary  1. 

SARP  in  which  you  work.  2. 

Which  of  the  following  3. 

services  are  provided  by 
this  SARP?  (MARK  ALL  4. 

THAT  APPLY) 


5. 


6. 


7. 

8. 

9. 

10. 
11. 
12. 

13. 

14. 

15. 


16. 


17. 


Screening  for  substance  use 
Screening  for  mental  health 
Comprehensive  substance 
abuse  assessment  or  diagnosis 
Comprehensive  mental  health 
assessment  or  diagnosis  (for 
example,  psychological  or 
psychiatric  evaluation  and 
testing) 

Outreach  to  persons  in  the 
community  who  might  need 
treatment 

Interim  services  for  cli¬ 
ents  when  admission  is  not 
possible 

Discharge  planning 
Aftercare/continuing  care 
Case  management 
Social  skills  development 
Mentoring/peer  support 
Substance  abuse  education 
Mental  health  education 
Self-help  groups 
Medication  Assisted  Therapy 
(e.g.,  Antabuse,  Naltrex¬ 
one,  Campral,  Methadone, 
Buprenorphine) 

Smoking  cessation  medica¬ 
tions  (nicotine  replacement  or 
non-nicotine) 

Medications  for  psychiatric 
disorders 


Counseling  For  each  type  of  counseling 

Type  below,  please  indicate 

approximately  what 
percentage  of  substance 
abuse  clients  at  this 
SARP  receive  that  type  of 
counseling  as  part  of  their 
substance  abuse  treatment 
program. 
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Table  C.1 — Continued 


Question 


Response  Option 


Individual  counseling 

1. 

Not  offered 

2. 

Received  by  25%  or  less  of 
clients 

3. 

Received  by  25%  to  50%  of 
clients 

4. 

Received  by  51%  to  75%  of 
clients 

5. 

Received  by  more  than  75% 
of  clients 

Group  counseling 

1. 

Not  offered 

2. 

Received  by  25%  or  less  of 
clients 

3. 

Received  by  25%  to  50%  of 
clients 

4. 

Received  by  51%  to  75%  of 
clients 

5. 

Received  by  more  than  75% 

of  clients 

Therapeutic  For  each  type  of  ciinicai/ 
approach  therapeutic  approach  iisted 
below,  please  mark  the  box 
that  best  describes  how 
often  that  approach  is  used 
at  this  SARP. 


Substance  use  counseling 

1. 

Not  familiar  with  this 
approach 

2. 

Never  or  rarely 

3. 

Sometimes 

4. 

Always  or  often 

12-step  facilitation 

1. 

Not  familiar  with  this 
approach 

2. 

Never  or  rarely 

3. 

Sometimes 

4. 

Always  or  often 

Supportive  counseling 

1. 

Not  familiar  with  this 
approach 

2. 

Never  or  rarely 

3. 

Sometimes 

4. 

Always  or  often 

Cognitive  behavioral 

1. 

Not  familiar  with  this 

therapy 

approach 

2. 

Never  or  rarely 

3. 

Sometimes 

4. 

Always  or  often 
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Table  C.1 — Continued 


Question 


Contingency  management 
or  motivational  incentives 


Motivational  interviewing 


Combined  cognitive 
behavioral  therapy  (CBT), 
motivational  interviewing 
(Ml),  12-Step  facilitation 
for  treating  co-occurring 
disorders 

Relapse  prevention 


Does  the  facility  offer 
a  special  program  for 
clients  with  co-occurring 
disorders? 


Response  Option 


1.  Not  familiar  with  this 
approach 

2.  Never  or  rarely 

3.  Sometimes 

4.  Always  or  often 

1.  Not  familiar  with  this 
approach 

2.  Never  or  rarely 

3.  Sometimes 

4.  Always  or  often 

1.  Not  familiar  with  this 
approach 

2.  Never  or  rarely 

3.  Sometimes 

4.  Always  or  Often 


1.  Not  familiar  with  this 
approach 

2.  Never  or  rarely 

3.  Sometimes 

4.  Always  or  Often 

0.  No 
1.  Yes 


Program  utility 


Self-efficacy/ 
achievement 
of  program 
objectives 


Please  rate  the  extent 
to  which  you  Agree 
or  Disagree  with  each 
statement.  (You  may  have 
answered  some  of  these 
questions  right  after  you 
finished  the  CODP;  we  are 
most  interested  in  how  you 
are  feeling  about  the  CODP 
now). 


As  a  result  of  attending 
the  CODP,  I  believe  I  am 
better  able  to  assess  seven 
areas  of  organizational 
effectiveness  presented 
in  the  CODP  clinical 
administrator's  guidebook. 


4.  Strongly  agree 
3. 

2. 

1.  Strongly  disagree 
Not  applicable  to  my  job 
responsibilities 
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Table  C.1 — Continued 


Question 


Response  Option 


As  a  result  of  attending 
the  CODP,  I  believe  I  am 
better  able  to  interpret 
comprehensive  guides  for 
assessing  an  organization's 
capacity  for  treating 
non-severe  co-occurring 
disorders:  Dual  Diagnosis 
Capability  in  Addiction 
Treatment  (DDCAT)  and 
Dual  Disorders  Capability  in 
Mentai  Health  Treatment 
(DDCMHT). 

As  a  result  of  attending 
the  CODP,  I  believe  I  am 
better  able  to  identify 
system,  program,  and  staff- 
level  issues  and  challenges 
in  providing  integrated 
services. 

As  a  result  of  attending 
the  CODP,  I  believe  I  am 
better  able  to  use  protocol- 
driven  screening  methods 
that  consider  each  person's 
symptoms,  history,  and 
motivation  to  change. 

As  a  result  of  attending 
the  CODP,  I  believe  I  am 
better  able  to  combine 
Motivational  Enhancement 
Therapy  (MET),  Cognitive 
Behavioral  Therapy  (CBT), 
and  Twelve-Step  Facilitation 
(TSF)  to  engage,  assist,  and 
sustain  change. 

As  a  result  of  attending  the 
CODP,  I  believe  I  am  better 
able  to  use  Cognitive- 
Behavioral  Therapy  to 
address  common  psychiatric 
problems,  such  as 
depression,  anxiety,  PTSD, 
and  others. 


4.  Strongly  agree 
3. 

2. 

1.  Strongly  disagree 
Not  applicable  to  my  job 
responsibilities 


4.  Strongly  agree 
3. 

2. 

1.  Strongly  disagree 
Not  applicable  to  my  job 
responsibilities 


4.  Strongly  agree 
3. 

2. 

1.  Strongly  disagree 
Not  applicable  to  my  job 
responsibilities 


4.  Strongly  agree 
3. 

2. 

1.  Strongly  disagree 
Not  applicable  to  my  job 
responsibilities 


4.  Strongly  agree 
3. 

2. 

1.  Strongly  disagree 
Not  applicable  to  my  job 
responsibilities 
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Table  C.1 — Continued 


Question 


Response  Option 


As  a  result  of  attending  the 
CODP,  I  believe  I  am  better 
able  to  examine  PTSD  and 
military  related  disorders, 
including  Traumatic  Brain 
Injury  (TBI). 

As  a  result  of  attending 
the  CODP,  I  believe  I  am 
better  able  to  convey  vital 
and  current  information 
on  medication  compliance, 
stigma,  and  other 
psychopharmacological 
issues. 


CODP  curricula 
and  materials: 
adoption  barriers, 
sustainability, 
fidelity 


Use  of  Which  of  the  following 

program  best  characterizes  how 

curricula  often  you  used  these 

CODP  materials  since  the 
training?  Exampies  of  "use" 
include  using  handouts 
or  measures  in  treatment 
or  assessment,  reading  or 
reviewing  the  manuals,  or 
using  content  to  guide  a 
session. 


Frequency  of  Administrator's  Guidebook 

use  of  each 

binder 


Screening  and  Assessment 


4.  Strongly  agree 

3. 

2. 

1.  Strongly  disagree 
Not  applicable  to  my  job 
responsibilities 

4.  Strongly  agree 
3. 

2. 

1.  Strongly  disagree 
Not  applicable  to  my  job 
responsibilities 


0.  Never 

1.  Once  or  twice 

2.  A  few  times  a  year 

3.  Monthly 

4.  Weekly 

0.  Never 

1.  Once  or  twice 

2.  A  few  times  a  year 

3.  Monthly 

4.  Weekly 
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Table  C.1 — Continued 


Question 


Response  Option 


Integrating  Combined 

0. 

Never 

Therapies 

1. 

Once  or  twice 

2. 

A  few  times  a  year 

3. 

Monthly 

4. 

Weekly 

Cognitive  Behavioral 

0. 

Never 

Therapy 

1. 

Once  or  twice 

2. 

A  few  times  a  year 

3. 

Monthly 

4. 

Weekly 

Medication  Management 

0. 

Never 

1. 

Once  or  twice 

2. 

A  few  times  a  year 

3. 

Monthly 

4. 

Weekly 

Quantity  of  Which  components  of  the 
use  of  specific  materials  do  you  use? 
materials 


Clinician's  guide 

0. 

Never 

1. 

Once  or  twice 

2. 

A  few  times  a  year 

3. 

Monthly 

4. 

Weekly 

Workbook  fact  sheets 

0. 

Never 

1. 

Once  or  twice 

2. 

A  few  times  a  year 

3. 

Monthly 

4. 

Weekly 

Workbook  handouts 

0. 

Never 

1. 

Once  or  twice 

2. 

A  few  times  a  year 

3. 

Monthly 

4. 

Weekly 

Screening/assessment 

0. 

Never 

measures 

1. 

Once  or  twice 

2. 

A  few  times  a  year 

3. 

Monthly 

4. 

Weekly 

The  next  items  ask  you  to 
consider  some  factors  that 
might  infiuence  your  use  of 
the  CODP  materials. 
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Table  C.1 — Continued 


Question 


Response  Option 


Adoption  You  were  satisfied  with 

of  program  the  materials  and  ideas 

materials  presented  in  the  materials. 


1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 


The  materials  have  proved 
to  be  relevant  to  the  needs 
of  your  clients  or  your 
program. 


You  have  been  comfortable 
using  them. 


You  have  found  the  things 
you  learned  are  useful  to 
you  and  your  clients/staff. 


Your  program  has  enough 
staff  capacity  to  implement 
these  materials. 


Your  program  has  adequate 
office  space  and  budget  to 
implement  these  materials. 


You  have  had  enough 
preparation  time  to  use 
these  materials. 


Others  in  your  program 
have  not  implemented 
these  materials  effectively. 


1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 
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Table  C.1 — Continued 


Question 


Counselors  in  your  program 
have  adequate  background 
and  training  needed  to  use 
these  materials. 


Practice  sessions  during 
the  training  gave  you 
confidence  in  using  the 
materials. 


The  training  provided  good 
instructions  and  examples 

for  adapting  the  materials 
to  your  clients'  needs. 


Based  on  what  you  learned, 
you  have  been  able  to 
train  others  to  use  these 
materials. 


A  follow-up  training  session 

is  needed  to  really  use 
these  materials  effectively. 


Your  program  director 
(or  clinical  supervisor)  has 
supported  and  encouraged 
use  of  these  materials. 


Other  staff  at  your  program 
have  become  interested 
in  learning  to  use  these 
materials. 


Staff  at  your  program  like 
to  help  one  another  when 
using  new  materials  like 
these. 


Response  Option 


1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 

1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 
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Table  C.1 — Continued 


Question 


Response  Option 


Your  clients/staff  benefited 

1. 

Disagree  Strongly 

from  and  encouraaed  vour 

2. 

Disagree 

use  of  the  materials. 

3. 

Undecided 

4. 

Agree 

5. 

Agree  Strongly 

You  have  found  ways  to 

1. 

Disagree  Strongly 

make  these  materials  a 

2. 

Disagree 

reaular  and  sustained  part 

3. 

Undecided 

of  your  program. 

4. 

Agree 

5. 

Agree  Strongly 

Barriers  to 

You  have  not  used  the 

using 

CODP  materials  more  often 

program 

because  ... 

materials 

...  you  have  a  lack  of  time. 

1. 

Disagree  Strongly 

2. 

Disagree 

3. 

Undecided 

4. 

Agree 

5. 

Agree  Strongly 

...  you  already  use  things 

1. 

Disagree  Strongly 

you  like  better. 

2. 

Disagree 

3. 

Undecided 

4. 

Agree 

5. 

Agree  Strongly 

...  they  do  not  fit  with  your 

1. 

Disagree  Strongly 

counseling  style. 

2. 

Disagree 

3. 

Undecided 

4. 

Agree 

5. 

Agree  Strongly 

...  your  agency  does  not 

1. 

Disagree  Strongly 

have  the  time  or  resources 

2. 

Disagree 

needed. 

3. 

Undecided 

4. 

Agree 

5. 

Agree  Strongly 

...  they  have  not  worked 

1. 

Disagree  Strongly 

with  your  clients. 

2. 

Disagree 

3. 

Undecided 

4. 

Agree 

5. 

Agree  Strongly 

Web  Survey  105 


Table  C.1 — Continued 


Question 


Response  Option 


...  you  do  not  feel  properly 
trained  to  use  them. 


1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 


...  they  seem  cumbersome 
and  difficult  to  use. 


1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 


...  they  do  not  comply  with 
the  treatment  philosophy 
at  your  agency. 


1.  Disagree  Strongly 

2.  Disagree 

3.  Undecided 

4.  Agree 

5.  Agree  Strongly 


Other:  Please  specify 


Attitudes 

Evidence-based  The  following  questions 

practices  ask  about  your  feelings 

attitudes  survey  about  using  new  types  of 
therapy,  interventions,  or 
treatments.  Manualized 
therapy,  treatment,  or 
intervention  refers  to 
any  intervention  that  has 
specific  guidelines  and/ 
or  components  that  are 
outlined  in  a  manual 
and/or  that  are  to  be 
followed  in  a  structured 
or  predetermined  way. 
Indicate  the  extent  to 
which  you  agree  with  each 
item. 


I  like  to  use  new  types  of 
therapy/interventions  to 
help  my  clients. 


0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 
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Table  C.1 — Continued 


Question 


Response  Option 


I  am  willing  to  try 
new  types  of  therapy/ 
interventions  even  if  I 
have  to  follow  a  treatment 
manual. 

I  know  better  than 
academic  researchers  how 
to  care  for  my  clients. 


I  am  willing  to  use  new  and 
different  types  of  therapy/ 
interventions  developed  by 
researchers. 


Research-based  treatments/ 
interventions  are  not 
clinically  useful. 


Clinical  experience  is  more 
important  than  using 
manualized  therapy/ 
interventions. 


I  would  not  use  manualized 
therapy/interventions. 


I  would  try  a  new  therapy/ 
intervention  even  if  it  were 
very  different  from  what  I 
am  used  to  doing. 


0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 

0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 

0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 


0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 

0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 

0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 

0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 


For  questions  9-15;  If 
you  received  training  in 
a  therapy  or  intervention 
that  was  new  to  you,  how 
likely  would  you  be  to 
adopt  it  if; 
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Table  C.1 — Continued 


Question 


Response  Option 


...  it  was  intuitively 
appealing? 


...  it  "made  sense"  to  you? 


...  it  was  required  by  your 
supervisor? 


...  it  was  required  by  your 
SARP  or  the  Navy? 


...  it  was  being  used  by 
colleagues  who  were  happy 
with  it? 


...  you  felt  you  had  enough 
training  to  use  it  correctly? 


0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 

0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 

0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 

0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 

0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 

0.  Not  at  all 

1.  To  a  slight  extent 

2.  To  a  moderate  extent 

3.  To  a  great  extent 

4.  To  a  very  great  extent 


Clinician 
attitudes 
towards  patients 
with  mental 
health  disorders 


Using  the  scale  beiow, 
please  indicate  how 
much  you  agree  with  the 
foliowing  statements. 


It  is  bad  for  people 

1. 

with  drug  and/or 

2. 

alcohol  problems  to  use 

3. 

medications. 

4. 

5. 

Strongly  Disagree 
Somewhat  Disagree 
Neither  Agree  nor  Disagree 
Somewhat  Agree 
Strongly  Agree 
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Table  C.1 — Continued 


Question 


Response  Option 


There  is  little  to  be  done  for  1. 
clients  that  are  mentally  ill.  2. 

3. 

4. 

5. 


It  is  easy  to  recognize  1. 

someone  who  has  a  mental  2. 

illness.  3. 

4. 

5. 

People  with  dual  diagnosis  1. 

should  get  better  with  2. 

abstinence  and  should  not  3. 

have  to  rely  on  medication.  4. 

5. 

People  with  mental  illness  1. 

have  weak  personalities.  2. 

3. 

4. 

5. 


Strongly  Disagree 
Somewhat  Disagree 
Neither  Agree  nor  Disagree 
Somewhat  Agree 
Strongly  Agree 

Strongly  Disagree 
Somewhat  Disagree 
Neither  Agree  nor  Disagree 
Somewhat  Agree 
Strongly  Agree 

Strongly  Disagree 
Somewhat  Disagree 
Neither  Agree  nor  Disagree 
Somewhat  Agree 
Strongly  Agree 

Strongly  Disagree 
Somewhat  Disagree 
Neither  Agree  nor  Disagree 
Somewhat  Agree 
Strongly  Agree 


Psychiatric  medications  are 
merely  a  replacement  for 
the  drug  of  choice. 


People  with  dual  diagnosis 
are  only  trying  to  get  on 
disability  payrolls. 


1.  Strongly  Disagree 

2.  Somewhat  Disagree 

3.  Neither  Agree  nor  Disagree 

4.  Somewhat  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Somewhat  Disagree 

3.  Neither  Agree  nor  Disagree 

4.  Somewhat  Agree 

5.  Strongly  Agree 


Brief  substance  Indicate  your  degree  of 

abuse  attitudes  agreement  or  disagreement 

survey  by  circling  the  appropriate 

choice  to  the  right  of  each 
statement.  There  are  no 
right  or  wrong  answers. 


Addiction  is  associated  with 
weak  will. 


1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 


Web  Survey  109 


Table  C.1 — Continued 


Question 


An  addicted  person  cannot 
be  helped  until  he/she  has 
hit  "rock  bottom." 


Heroin  is  so  addicting  that 
no  one  can  really  recover 
once  he/she  becomes  an 
addict. 


Alcohol  and  drug  abusers 
should  only  be  treated  by 
specialists  in  that  field. 


Smoking  leads  to  marijuana 
use,  which  in  turn  leads  to 
hard  drugs. 


Professionals  who  diagnose 
alcoholism  early  improve 
the  chance  of  treatments 
success. 


Daily  use  of  one  marijuana 
cigarette  is  not  necessarily 
harmful. 


Urine  drug  screening  can  be 
an  important  part  of  drug 
abuse  treatment. 


A  professional  who  has 
been  addicted  to  narcotics 
should  be  allowed  to 
practice  within  their  field 
again. 


Response  Option 


1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 
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Table  C.1 — Continued 


Question 


Marijuana  use  among 
teenagers  can  be  healthy 
experimentation. 


An  addicted  person  who 
has  relapsed  several  times 
probably  cannot  be  treated. 


Long-term  outpatient 
treatment  is  necessary  for 
the  treatment  of  addiction. 


Paraprofessional  counselors 
can  provide  effective 
treatment  for  addicted 
individuals. 


Lifelong  abstinence  is 
a  necessary  goal  in  the 
treatment  of  addiction. 


Once  a  person  becomes 
chemical-free  through 
treatment,  she  or  he  can 
never  become  a  social  user. 


Drug  addiction  is  a 
treatable  disease. 


Group  therapy  is  very 
important  to  the  treatment 
of  addiction. 


Response  Option 


1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 
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Table  C.1 — Continued 


Question 


A  hospital  is  the  best  place 
to  treat  an  alcohol  or  drug 
addict. 


Alcoholism  is  a  treatable 
disease. 


Most  addicted  persons  are 
unpleasant  to  work  with  as 
patients. 


Pregnant  women  who  use 
alcohol  and  other  drugs 
should  be  punished. 


Coercive  pressure,  such 
as  threat  of  punishment, 
is  useful  in  getting 
resistant  patients  to  accept 
treatment. 

A  recovering  person  who 
is  active  in  Alcoholics 
Anonymous  does 
not  respond  well  to 
psychotherapy. 

A  professional  who  has 
been  drug  dependent 
should  not  be  allowed 
to  give  medications  to 
patients. 

Active  participation  in 
a  program  such  as  AA  is 
essential  for  a  patient  to 
recover  from  alcohol  or 
drug  dependence. 


Response  Option 


1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 
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Table  C.1 — Continued 


Question 


I  feel  comfortable  in 
interviewing  a  patient  with 
substance  abuse  problems. 


I  feel  able  to  take  a 
complete  history  from  a 
patient  with  substance 
abuse  problems. 


I  feel  competent  to  make  a 
diagnosis  of  alcohol/drug 
abuse  or  dependence. 


I  feel  equipped  to 
recommend  appropriate 
treatment  referrals  for 
substance  abuse  patients. 


Response  Option 


1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 

1.  Strongly  Disagree 

2.  Disagree 

3.  Uncertain 

4.  Agree 

5.  Strongly  Agree 


Dual-diagnosis 

capability 

DDCAT 

assessment 


When  you  attended 
the  CODP  training, 
the  Dual  Diagnosis 
Capability  in  Addiction 
Treatment  (DDCAT)  was 
introduced.  The  DDCAT 
is  a  rating  system  that 
clinic  administrators  can 
use  to  assess  their  clinic's 
capability  to  serve  dual 
diagnosis  clients.  We  are 
presenting  the  DDCAT 
again.  Within  each  domain 
please  select  the  response 
that  best  applies  to  your 
program  or  agency. 
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Table  C.1 — Continued 


Program 

structure 


Question 


What  is  the  primary  focus  1. 

ofyourSARP?  2. 


3. 


What  does  your  SARP's  1. 

organizationai  certification 
and  iicensure  permit?  2. 


3. 


4. 


To  what  extent  does  your  1 . 

SARP  coordinate  and 
coiiaborate  with  mentai 
heaith  services?  2. 


3. 


4. 


5. 


Response  Option 


Addiction  oniy. 

Primary  focus  is  addiction, 
but  co-occurring  disorders  are 
treated. 

Primary  focus  on  persons  with 
co-occurring  disorders. 

Permits  oniy  addiction 
treatment. 

Has  no  actuai  barrier,  but 
staff  report  there  to  be  certi¬ 
fication  or  iicensure  barriers. 
Has  no  barrier  to  providing 
mentai  heaith  treatment  or 
treating  co-occurring  disor¬ 
ders  within  the  context  of 
addiction  treatment, 
is  certified  and/or  iicensed  to 
provide  both  mentai  heaith 
and  addiction  treatment. 

There  is  no  formai  or  docu¬ 
mented  coordination  or 
coiiaboration. 

There  is  vague,  undocu¬ 
mented  or  informai  reiation- 
ship  with  mentai  heaith  agen¬ 
cies,  or  consuiting  with  a  staff 
member  from  that  agency. 
There  is  formaiized  and 
documented  coordination 
or  coiiaboration  with  mentai 
heaith  agency. 

There  is  formaiized  coordi¬ 
nation  &  coiiaboration,  and 
aiso  the  avaiiabiiity  of  case 
management  staff,  or  staff 
exchange  programs. 

Most  mentai  heaith  services 
are  integrated  within  the 
existing  program,  or  there  is 
routine  use  of  case  manage¬ 
ment  staff  or  staff  exchange 
programs. 
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Table  C.1 — Continued 


Question 


Response  Option 


Program 

milieu 


What  is  the  billing  structure  1. 
at  your  SARP?  Can  you  bill 
for  mental  health  disorders, 
substance  use  disorders,  or  2. 
both? 


3. 


4. 


Are  patients  with  mental  1. 
health  disorders  expected 
and  welcomed  at  your 
SARP? 

2. 


3. 


4. 


We  can  only  bill  for  addiction 
treatments  for  persons  with 
substance  use  disorders. 

We  could  bill  for  either  type  if 
substance  use  disorder  is  pri¬ 
mary,  but  staff  report  there 
to  be  barriers.  -OR-  Partial 
reimbursement  for  mental 
health  services  are  available. 
We  can  bill  for  either  service 
type;  however,  substance  use 
disorder  must  be  primary. 

We  can  bill  for  addiction  or 
mental  health  treatments 
or  the  combination  and/or 
integration. 

My  SARP  expects  substance 
use  disorders  only.  We  refer 
or  deflect  persons  with 
mental  health  disorders  or 
symptoms. 

My  SARP  is  documented 
to  expect  substance  use 
disorders  only  (e.g.  admis¬ 
sion  criteria,  target  popula¬ 
tion),  but  we  have  informal 
procedure(s)  to  allow  some 
persons  with  mental  health 
problems  to  be  admitted. 

We  expect  substance  use  dis¬ 
orders,  and,  with  documen¬ 
tation,  accept  mental  health 
disorders  by  routine  and  if 
mild  and  relatively  stable. 

Our  program  is  formally 
defined  such  that  we  only 
expect  substance  use  disor¬ 
ders,  but  our  clinicians  and 
program  informally  expect 
and  treat  both  disorders. 

Our  clinicians  and  program 
expect  and  treat  both  dis¬ 
orders,  and  this  is  well 
documented. 
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Table  C.1 — Continued 


Question 


Response  Option 


What  is  the  focus  of  patient  1. 
literature  and  education 
materials  in  your  SARP? 

2. 


3. 


4. 


5. 


Only  addiction  or  peer  sup¬ 
port  (e.g.,  AA)  materials  are 
displayed  and  distributed. 
Materials  are  available  for 
both  disorders  but  not  rou¬ 
tinely  offered  or  formally 
available. 

Materials  are  available  for 
both  mental  health  &  sub¬ 
stance  use  disorders,  but 
distribution  is  less  for  mental 
health  problems. 

Materials  are  available  for 
both  mental  health  &  sub¬ 
stance  use  disorders  with 
equivalent  distribution. 
Materials  are  available  for  the 
interaction  between  mental 
health  and  substance  use 
disorders. 


Clinical 

process: 

assessment 


How  are  psychiatric 
symptoms  screened  in  your 
SARP? 


1.  Pre-admission  mental  health 
screening  depends  on  patient 
self-report.  The  decision 
whether  to  screen  is  based 
on  clinician  inference  from 
patient  presentation  or  by 
history. 

2.  Pre-admission  mental  health 
screening  and  symptom  & 
treatment  history,  current 
psychiatric  medications,  and/ 
or  suicide/homicide  his¬ 
tory  are  conducted  prior  to 
admission. 

3.  There  is  a  routine  set  of  stan¬ 
dard  interview  questions 
for  mental  health  using  a 
generic  framework,  such  as 
the  ASAM-PPC  (Dimension 
III)  or  "Biopsychosocial"  data 
collection. 
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Table  C.1 — Continued 


Question 


Response  Option 


4. 


5. 


What  is  the  assessment  1. 

routine  if  a  patient  screens 
positive  for  psychiatric 
symptoms? 

2. 


3. 


4. 


5. 


To  what  extent  are  1 . 

psychiatric  and  substance 
use  diagnoses  made  and  2. 

documented  at  your  SARP? 

3. 


4. 


5. 


We  screen  for  mental  health 
problems  using  standardized 
or  formal  instruments  with 
established  psychometric 
properties. 

We  use  standardized  or 
formal  instruments  for  both 
mental  health  and  substance 
use  disorders  with  established 
psychometric  properties. 

There  is  ongoing  monitoring 
of  psychiatric  symptoms  to 
determine  patient  appropri¬ 
ateness  or  exclusion  from  the 
program. 

Clinicians  conduct  a  detailed 
biopsychosocial  assessment 
and/or  mental  status  exam. 
We  conduct  formal  mental 
health  assessments,  if 
necessary. 

We  conduct  or  obtain  follow¬ 
up  mental  health  assess¬ 
ments,  although  these  are 
not  standardized  or  routine. 
We  use  standardized  or 
formal  integrated  mental 
health  and  substance  use 
assessment  in  all  cases. 

Psychiatric  diagnoses  are  not 
made  or  recorded. 

Mental  health  diagnostic 
impressions  are  made  and 
recorded  variably. 

Mental  health  diagnosis 
is  variably  recorded  in  the 
patient's  chart. 

Mental  health  diagnosis  is 
frequently  recorded,  but 
inconsistently. 

Standard  &  routine  mental 
health  diagnoses  are  consis¬ 
tently  made. 
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Table  C.1 — Continued 


Question 


Response  Option 


How  are  patients'  1. 

psychiatric  and  substance 
use  histories  reflected  in  2. 

their  medical  records? 


3. 


4. 


5. 


What  level  of  psychiatric  1. 

symptom  acuity  is  required 
to  be  admitted  into  your 
SARP?  Acuity  refers  to  the  2. 

level  of  need,  or  urgency, 
of  the  symptoms. 

3. 


What  level  of  psychiatric  1. 

symptom  severity  and 
persistence  is  required  to  be 
admitted  into  your  SARP? 

2. 


3. 


We  collect  substance  use  dis¬ 
order  history  only. 

Our  standard  protocol  col¬ 
lects  substance  use  disor¬ 
der  history  only.  Mental 
health  history  is  collected 
inconsistently. 

We  routinely  document  both 
mental  health  and  substance 
use  disorder  history  in  the 
narrative  section  of  patient 
records. 

There  is  a  specific  section  in 
our  patients'  records  dedi¬ 
cated  to  history  and  chro¬ 
nology  of  course  of  both 
disorders. 

There  is  a  specific  section  in 
records  devoted  to  history 
and  chronology  of  course 
of  both  disorders  and  the 
interaction  between  them  is 
examined  over  time. 

We  admit  people  with  no 
to  low  psychiatric  symptom 
acuity. 

We  admit  people  with  low  to 
moderate  acuity  but  who  are 
primarily  stable. 

We  admit  people  with  mod¬ 
erate  to  high  acuity,  including 
those  unstable  in  their  psychi¬ 
atric  condition. 

We  admit  people  into  our 
program  who  have  no  to  low 
severity  or  persistence  of 
mental  health  disability. 

We  admit  people  into  our 
program  with  low  to  mod¬ 
erate  psychiatric  symptom 
severity. 

We  admit  people  into  our 
program  with  moderate  to 
high  psychiatric  symptom 
severity. 
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Table  C.1 — Continued 


Question 


Response  Option 


Does  your  SARP  assess  1. 

patients'  stages  of  change? 

2. 


3. 


4. 


5. 


Stages  of  change  are  not 
assessed  or  documented. 
Stages  of  change  are  assessed 
&  documented  variably 
depending  on  the  individual 
clinician. 

Clinicians  assess  and  rou¬ 
tinely  document  stages  of 
change,  which  are  focused 
on  substance  use  disorders 
motivation. 

Formal  measures  are  used 
and  stages  of  change  are  rou¬ 
tinely  documented,  but  focus 
on  substance  use  disorders 
motivation  only. 

Formal  measures  are  used 
and  stages  of  change  are  rou¬ 
tinely  documented,  and  focus 
on  both  substance  use  and 
mental  health  motivation. 


Clinical 

process: 

treatment 


What  is  the  focus  of  1. 

treatment  plans  in  your 
SARP?  2. 

3. 


4. 


5. 


How  does  your  SARP  assess  1. 

and  monitor  the  courses  of 
substance  use  and  mental  2. 

health  disorders? 


Our  treatment  plans  address 
addiction  only. 

It  depends  on  the  individual 
clinician. 

In  our  treatment  plans,  sub¬ 
stance  use  disorders  are 
addressed  as  the  primary 
problem,  with  mental  health 
as  secondary. 

Our  plans  routinely  address 
both  substance  use  and 
mental  health  disorders,  but 
the  amount  of  detail  for  each 
type  of  disorder  varies. 

Our  treatment  plans  address 
both  substance  use  and 
mental  health  disorders  as 
primary,  and  both  are  listed 
in  plans  consistently. 

We  do  not  track  progress 
with  mental  health  problems. 
Report  of  progress  on  mental 
health  problems  varies 
depending  on  the  individual 
clinician. 
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Table  C.1 — Continued 


Question 


Response  Option 


3. 


4. 


5. 


How  are  procedures  for  1. 

psychiatric  emergencies 
and  crisis  management 
conveyed  in  your  SARP?  2. 

3. 


4. 


5. 


To  what  extent  does  1 . 

your  SARP  use  stage-wise 
treatment? 

2. 


Clinicians  routinely  note 
changes  in  mental  health 
problems  in  their  treatment 
plans  and  progress  notes. 

We  have  tools  available 
for  clinicians  to  systemati¬ 
cally  monitor  and  document 
changes  in  both  substance 
use  and  mental  health  disor¬ 
ders,  but  use  of  these  tools 
depends  on  the  clinician. 

We  consistently  and  systemat¬ 
ically  focus  on  change  in  both 
substance  use  and  mental 
health  disorders  in  a  detailed 
fashion. 

No  guidelines  for  psychiatric 
emergencies  are  conveyed  in 
any  manner. 

In-house  guidelines  are  ver¬ 
bally  conveyed. 

We  have  documented  guide¬ 
lines  that  suggest  referral  to 
or  collaboration  with  others 
(e.g.,  local  mental  health 
agency  or  ER). 

We  have  documented  guide¬ 
lines,  risk  assessment  tools, 
and  advance  directives,  but 
these  are  not  consistently 
used. 

We  have  documented  guide¬ 
lines,  risk  assessment  tools, 
and  advance  directives,  and 
they  are  consistently  used. 

We  keep  patients  in  our  pro¬ 
gram  unless  inpatient  hospi¬ 
talization  is  warranted. 

Stages  of  change  are  not 
assessed  or  explicit  in  treat¬ 
ment  plans. 

Stage  of  change  or  motiva¬ 
tion  are  sometimes  docu¬ 
mented  in  treatment  plans  by 
some  clinicians. 
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Table  C.1 — Continued 


Question 


Response  Option 


3. 


4. 


5. 


What  are  your  policies  1. 

and  procedures  for 
medication  evaluation,  2. 

management,  monitoring, 
and  compliance? 


3. 


Stage  of  change  or  motiva¬ 
tion  are  routinely  incorpo¬ 
rated  into  individualized 
plans,  but  no  specific  stage- 
wise  treatments  are  provided. 
Stage  of  change  or  motiva¬ 
tion  are  routinely  incorpo¬ 
rated  into  individualized 
plans,  and  there  is  a  general 
awareness  of  the  value  of 
adjusting  treatments  by  sub¬ 
stance  use  stage  of  change  or 
motivation. 

Stage  of  change  or  motiva¬ 
tion  are  routinely  incorpo¬ 
rated  into  individualized 
plans,  and  stage-wise  treat¬ 
ments  are  formally  prescribed 
and  delivered  for  both  sub¬ 
stance  use  and  mental  health 
issues. 

Patients  on  meds  are  typically 
not  accepted. 

Certain  types  of  meds  are  not 
acceptable,  or  the  patient 
must  have  his/her  own  supply 
for  the  entire  treatment  epi¬ 
sode.  We  have  some  capac¬ 
ity  to  monitor  psychotropic 
medications. 

Patients  have  some  access 
to  a  prescriber  for  psycho¬ 
tropic  medications,  and  we 
have  policies  to  guide  pre¬ 
scribing  within  the  program. 
Monitoring  of  the  medica¬ 
tion  is  largely  done  by  the 
prescriber. 
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Table  C.1 — Continued 


Question 


Response  Option 


4.  Our  medicating  provider 
is  a  staff  member  who  has 
clear  standards  and  routines. 
Patients  have  regular  access 
to  the  prescriber  and  guide¬ 
lines  for  prescribing  are  in 
place.  The  prescriber  some¬ 
times  consults  with  other 
staff  regarding  the  medica¬ 
tion  plan  and  recruits  other 
staff  to  assist  with  medication 
monitoring. 

5.  Our  medicating  provider  is 
also  a  staff  member  and  is 
present  on  treatment  teams 
or  in  the  SARP  administra¬ 
tion.  S/he  has  clear  standards 
and  routines.  Patients  have 
full  access  to  the  prescriber 
and  appropriate  prescribing 
guidelines  are  in  place.  As  a 
treatment  team  member,  the 
prescriber  informs  the  team 
about  the  medication  plan 
and  the  entire  team  can  assist 
with  monitoring. 


Are  there  specialized 

1. 

Mental  health  is  not 

mental  health  interventions 

addressed  in  our  program 

in  your  SARP? 

content. 

2. 

Incorporation  of  mental 
health  material  is  up  to  the 
individual  clinician. 

3. 

We  have  generalized  mental 
health  interventions. 

(e.g.,  stress  management). 
Clinicians  routinely  provide 
evidence-based  addiction 
treatment  (e.g.,  Motivational 
Interviewing,  CBT,  TSF)  that 
may  have  mental  health 
content. 
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Table  C.1 — Continued 


Question 


Response  Option 


4. 


5. 


To  what  extent  is  education  1. 

about  psychiatric  disorders,  2. 

their  treatment,  and  their  3. 

interaction  with  substance 
use  &  its  treatment  offered 
atyourSARP?  4. 


5. 


To  what  extent  does  your  1 . 

SARP  offer  family  education 
and  support? 

2. 


3 


In  addition  to  routine  gener¬ 
alized  interventions,  we  have 
some  specialized  interven¬ 
tions  provided  by  specifically 
trained  clinicians. 

We  routinely  offer  mental 
health  symptom  management 
groups.  We  have  individual 
therapies  focused  on  specific 
disorders,  and  systematically 
use  an  evidence-based  addic¬ 
tion  treatment  (e.g..  Motiva¬ 
tional  Interviewing,  CBT,  TSF). 

Not  offered 

This  is  sometimes  offered. 

We  offer  general  informa¬ 
tion,  delivered  in  individual 
and/or  group  formats. 

We  sometimes  provide  educa¬ 
tion  in  specific  content  areas 
in  some  group  or  individual 
settings. 

We  offer  specific  content  for 
specific  disorder  co-morbidi¬ 
ties,  and  deliver  it  in  individ¬ 
ual  and/or  group  formats. 

We  offer  this  for  alcohol  or 
drug  problems  only,  or  not  at 
all. 

We  sometimes  offer  this, 
depending  on  individual  clini¬ 
cal  judgment. 

Mental  health  issues  are 
regularly  but  informally  incor¬ 
porated  into  family  education 
or  support  sessions,  which  are 
available  as  needed. 
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Table  C.1 — Continued 


Question 


Response  Option 


4. 


5. 


Does  your  SARP  have  1. 

specialized  interventions 
to  facilitate  the  use  of  peer 
support  groups? 

2. 


3. 


4. 


We  sometimes  offer  generic 
groups  on  substance  use 
and  mental  issues  on  site  for 
families. 

We  routinely  and  system¬ 
atically  provide  co-occurring 
disorder  family  groups,  which 
are  integrated  into  our  stan¬ 
dard  program  format. 

We  do  not  use  interventions 
to  facilitate  the  use  of  either 
addiction  or  mental  health 
peer  support. 

These  interventions  are  some¬ 
times  used  by  individual  clini¬ 
cians  for  individual  patients, 
mostly  for  facilitation  of 
addiction  peer  support 
groups. 

We  routinely  facilitate  tra¬ 
ditional  addiction,  but  not 
mental  health,  peer  sup¬ 
port  groups  (e.g..  Alcohol¬ 
ics  Anonymous,  Narcotics 
Anonymous). 

We  sometimes  facilitate  peer 
support  groups  targeting 
specific  mental  health  issues, 
either  traditional  peer  sup¬ 
port  groups  or  those  specific 
to  both  substance  use  and 
mental  health  (e.g..  Dual  Re¬ 
covery  Anonymous,  Double 
Trouble  in  Recovery,  etc.). 

We  routinely  and  specifi¬ 
cally  facilitate  peer  support 
groups  to  address  the  needs 
of  co-occurring  persons.  We 
have  special  programs  on  site, 
which  are  routinely  targeted 
to  specific  issues. 
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Table  C.1 — Continued 


Question 


Response  Option 


Are  peer  recovery  supports  1. 
available  for  patients  with 
co-occurring  disorders? 

2. 


3. 


4. 


5. 


We  do  not  have  such  groups; 
if  these  groups  are  available, 
they  are  not  recommended. 
There  are  groups  off  site, 
and  they  are  sometimes 
recommended. 

There  are  groups  facilitated 
off  site.  We  work  with  a  con¬ 
tact  person  or  use  an  infor¬ 
mal  matching  process  with 
peer  supports  in  the  commu¬ 
nity.  The  groups  have  some 
co-occurring  focus. 

Groups  are  available  off  site, 
are  integrated  into  patients' 
treatment  plans,  and  have  a 
co-occurring  focus. 

Groups  are  available  on  site, 
and  are  facilitated  and  inte¬ 
grated  into  our  program  (e.g. 
alumni  groups);  these  groups 
are  routinely  used  and  have  a 
co-occurring  focus. 


Continuity  of 

To  what  extent  are  co¬ 

1. 

Co-occurring  disorders  are 

care 

occurring  disorders 

not  addressed  in  discharge 

addressed  in  the  discharge 

planning. 

planning  process  at  your 

2. 

Co-occurring  disorders  are 

SARP? 

sometimes  addressed  by  cer¬ 
tain  clinicians. 

3. 

Co-occurring  disorders  are 

systematically  addressed  as 
secondary  in  the  planning 
process  for  offsite  referral. 

4.  We  address  both  substance 
use  and  mental  health  dis¬ 
orders  as  priorities  in  our 
discharge  plans  a  lot  but  not 
always  (e.g.,  less  than  80%  of 
the  time). 

5.  Both  mental  health  and  sub¬ 
stance  use  disorders  are  seen 
as  primary  in  discharge  plans, 
and  we  make  plans  for  on- 
or  off-site  follow-up  at  least 
80%  of  the  time. 
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Table  C.1 — Continued 


Question 


Response  Option 


Does  your  program  have  1. 

the  capacity  to  provide 
continuing  care  for  co¬ 
occurring  disorders  after 
addiction  treatment? 

2. 


3. 


4. 


5. 


Does  your  program  focus  1. 

on  ongoing  recovery  issues  2. 

for  both  substance  use  and 
mental  health  disorders?  3. 


4. 


We  have  no  mechanism  for 
managing  ongoing  care  of 
mental  health  needs  when 
the  addiction  treatment  pro¬ 
gram  is  completed. 

We  have  no  formal  protocol 
to  manage  mental  health 
needs  once  the  program  is 
completed,  but  some  indi¬ 
vidual  clinicians  may  provide 
extended  care  until  appropri¬ 
ate  linkage  takes  place. 

No  formal  protocol  is  in  place 
to  manage  mental  health 
needs  once  the  program  is 
completed.  When  indicated, 
however,  most  individual 
clinicians  provide  extended 
care  until  appropriate  linkage 
takes  place. 

We  have  a  formal  protocol  to 
manage  mental  health  needs 
indefinitely,  but  this  is  not 
routinely  practiced. 

We  have  a  formal  protocol  to 
manage  mental  health  needs 
indefinitely  and  this  is  rou¬ 
tinely  practiced. 

No. 

This  depends  on  the  individ¬ 
ual  clinician. 

We  routinely  focus  on  recov¬ 
ery  from  addiction,  and 
mental  health  issues  are 
viewed  only  as  potential 
relapse  issues. 

We  routinely  focus  on  addic¬ 
tion  recovery  and  mental 
health  illness  management 
and  recovery.  Both  are  seen 
as  primary  and  ongoing. 
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Table  C.1 — Continued 


Question 


Response  Option 


Do  you  have  specialized  1. 

interventions  to  facilitate 
use  of  community-based 
peer  support  groups  during 
discharge  planning? 

2. 


3. 


4. 


5. 


Do  you  have  a  supply  1. 

and  compliance  plan  for 
medications?  2. 


3. 


4. 


5. 


We  have  no  interventions 
to  facilitate  use  of  either 
addiction  or  mental  health 
peer  support  groups  upon 
discharge. 

Individual  clinicians  some¬ 
times  address  this,  mostly 
for  facilitation  to  addiction 
peer  support  groups  upon 
discharge. 

We  have  some  generic  inter¬ 
ventions,  but  no  specific  or 
intentional  facilitation  based 
on  mental  health  disorders. 
We  have  more  routine  facili¬ 
tation  to  addiction  peer  sup¬ 
port  groups  (e.g..  Alcoholics 
Anonymous,  Narcotics  Anony¬ 
mous)  upon  discharge. 

We  sometimes  provide  inter¬ 
ventions  targeting  specific  co¬ 
occurring  needs  to  facilitate 
use  of  addiction  or  co-occur- 
ring  disorders  peer  support 
groups  upon  discharge. 

We  routinely  and  systemati¬ 
cally  conduct  such  interven¬ 
tions,  with  a  focus  on  co¬ 
occurring  needs  to  facilitate 
use  of  peer  support  groups 
for  addiction  or  co-occurring 
disorders  after  discharge. 

We  have  no  such  medications 
plan. 

Sometimes  we  have  a  30-day 
medication  plan,  or  can 
supply  medication  until  the 
next  off-site  appointment. 

We  routinely  have  a  30-day 
plan  or  supply  to  next 
appointment  off-site. 
Medication  management 
is  maintained  within  our 
program  until  the  patient  is 
admitted  to  the  next  level  of 
care  at  a  different  provider. 
Medication  management  is 
maintained  in  our  program. 
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Table  C.1 — Continued 


Question 


Response  Option 


Staffing 


Does  your  SARP  have  1. 

a  psychiatrist  or  other 
physician  or  prescriber  of 
psychotropic  mecJications?  2. 

3. 

4. 


5. 


What  portion  of  your  1. 

SARP's  on-site  clinical  staff  2. 

members  have  mental 
health  licensure  (doctoral  3. 

or  master's  level),  or  similar 
competency/experience?  4. 


5. 


Do  your  staff  have  access  to  1. 
mental  health  supervision/  2. 

consultation? 

3. 


4. 


5. 


We  have  no  formal  relation¬ 
ship  with  a  prescriber  for  this 
program. 

We  work  with  a  consultant  or 
contractor  off  site. 

We  work  with  a  consultant  or 
contractor  on  site. 

We  have  a  staff  member  to 
do  this.  S/he  is  present  on  site 
for  clinical  matters  only. 

We  have  a  staff  member 
to  do  this.  S/he  is  present 
on  site  for  clinical,  supervi¬ 
sion,  treatment  team,  and/or 
administration. 

None. 

1-24%  of  our  clinical  staff 
members. 

25-33%  of  clinical  staff 
members. 

34-49%  of  clinical  staff 
members. 

50%  or  more  of  clinical  staff 
members. 

No. 

Our  staff  have  access  to 
off-site  supervision  by  a 
consultant. 

On-site  supervision  is  pro¬ 
vided  as  needed  by  a  con¬ 
sultant,  contractor,  or  staff 
member. 

On  site  supervision  is  pro¬ 
vided  regularly  by  a  staff 
member. 

On  site  supervision  is  rou¬ 
tinely  provided  by  a  staff 
member  and  focuses  on  in- 
depth  learning. 
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Table  C.1 — Continued 


Question 


Response  Option 


Does  your  SARP  conduct 
case  review,  staffing, 
or  utilization  review 
procedures  that  emphasize 
and  support  co-occurring 
disorder  treatment? 


Is  there  peer/alumni 
support  available  for  co¬ 
occurring  disorders? 


4. 


5. 


1.  No,  these  are  not  conducted. 

2.  These  are  sometimes  con¬ 
ducted  by  an  off-site 
consultant. 

3.  These  are  conducted  on  site, 
documented,  and  cover  co¬ 
occurring  disorder  issues  as 
needed. 

4.  These  are  routinely  con¬ 
ducted  and  documented,  but 
do  not  systematicaliy  cover 
co-occurring  issues. 

5.  These  are  routinely  con¬ 
ducted  and  documented,  and 
systematically  cover  co-occur- 
ring  issues. 

1.  Not  available. 

2.  These  supports  are  available, 
but  as  part  of  the  commu¬ 
nity.  Some  clinicians  might 
refer  their  patients  to  these 
supports. 

3.  These  supports  are  available, 
but  as  part  of  the  community. 
Referrals  are  made  routinely 
through  clinician  relation¬ 
ships  or  more  formal  connec¬ 
tions  such  as  peer  support 
service  groups  (e.g..  Alcohol¬ 
ics  Anonymous  hospital  and 
institutional  committees; 
National  Alliance  on  Mental 
Illness). 

These  supports  are  avail¬ 
able  on  site,  either  as  paid 
staff,  volunteers,  or  program 
alumni.  Referrals  are  some¬ 
times  made. 

Referrals  are  routinely  made 
to  these  supports,  which  are 
available  on  site,  either  as 
paid  staff,  volunteers,  or  pro¬ 
gram  alumni. 
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Table  C.1 — Continued 


Question 


Response  Option 


Training 


What  portion  of  your  1. 

SARP's  staff  members  have 
basic  training  in  attitudes.  2. 

prevalence,  common  signs 
&  symptoms,  detection, 
and  triage  for  co-occurring  3. 

disorders? 


4. 


5. 


What  portion  of  your  1. 

SARP's  staff  members 
have  advanced  specialized  2. 

training  in  integrated 
or  psychosocial  or 
pharmacological  treatment  3. 

of  people  with  co-occurring 
disorders? 


4. 


5. 


No  staff  have  basic  training 
(0%  trained). 

Some  are  trained,  but  we 
have  no  systematic  training 
plan  (1-24%  of  staff  trained). 
Certain  staff  are  trained  with 
a  systematic  training  plan, 
and  they  are  encouraged  by 
management  (25-50%  of 
staff  trained). 

Many  staff  are  trained  and 
monitored  by  our  program's 
strategic  training  plan  (51- 
79%  of  staff  trained). 

Most  staff  are  trained,  and 
are  periodically  monitored 
by  our  program's  strategic 
training  plan  (80%  or  more  of 
staff  trained). 

No  staff  have  advanced  train¬ 
ing  (0%  trained). 

Some  are  trained,  but  we 
have  no  systematic  training 
plan  (1-24%  of  staff  trained). 
Certain  staff  are  trained  with 
a  systematic  training  plan, 
and  they  are  encouraged  by 
management  (25-50%  of 
staff  trained). 

Many  staff  are  trained  and 
monitored  by  our  program's 
strategic  training  plan  (51- 
79%  of  staff  trained). 

Most  staff  are  trained,  and 
are  periodically  monitored 
by  our  program's  strategic 
training  plan  (80%  or  more  of 
staff  trained). 


We  are  conducting 
telephone  interviews 
with  a  select  number  of 
survey  respondents  so 
that  we  can  get  more  in 
depth  information  about 
implementation  of  the 
training  program.  We 
expect  the  interviews  will 
take  up  to  45  minutes. 
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Question 


Response  Option 


Are  you  interested  in 
learning  more  about 
participating  in  an 
interview? 


0.  No 
1.  Yes 


IF  INTERVIEW  =  1  Thank  you  for  participating 
in  this  study.  Your 
responses  will  help 
us  to  improve  care  for 
individuals  with  co¬ 


occurring  disorders.  If  you 
are  selected  as  a  potential 
participant  for  a  telephone 
interview,  we  will  contact 
you  via  email  within  a  few 
weeks. 


IF  INTERVIEW  =  0  Thank  you  for  participating 


in  this  study.  Your 
responses  will  help  us  to 
improve  care  for  individuals 
with  co-occurring 
disorders. 


APPENDIX  D 

Staff  Discussion  Guide 


Table  D.1 

Staff  Discussion  Guide 


Topic 

Question  to 

Initiate  Discussion  Possible  Prompts/Probes 

Program  utility 

Tell  me  a  bit  about  your  experience  participating  in  the 
CODP  program. 

Program  met  CODP 
goal 

Do  you  think  the  program  met  its  goals 
of ... 

what  were  personal  What  were  your  goals  for  attending  the 

goals  CODP? 


Program  met 
personal  goals 

Did  CODP  meet  these  goals?  Why?  Why 
not? 

Program  met  CODP 
objectives 

Can  you  talk  about  how  well  the 

CODP  expanded  the  capability  of 
your  program  to  treat  clients  with  co¬ 
occurring  substance  use  and  mental 
health  disorders? 

CODP  curricula 
and  materials: 
implementation, 
sustainability,  barriers 

During  the  CODP  training,  you  designed  an  Action  Plan 
describing  the  specific  ways  you  intended  to  use  the  CODP 
materials  for  intake  and  assessment,  care/treatment,  and 
relapse  prevention/continuing  care  with  your  clients.  Of 
course,  this  was  a  long  time  ago  and  things  may  have 
changed  a  lot  since  then.  Let's  look  at  your  Action  Plan 
together  and  I'd  like  to  ask  you  some  questions  about 
which  components  you  implemented,  which  you  didn't, 
and  for  what  reasons. 

Intake  and  assessment 
phase 

You  mentioned  in  your  Action  Plan  that  you  wanted  to  [fill 
in  with  actual  text  from  individual's  plan]  to  improve  intake 
and  assessment  for  clients  with  co-occurring  disorders. 
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Topic 

Question  to 

Initiate  Discussion  Possible  Prompts/Probes 

Implementation  of 
Action  Plan 

How  regularly  do  you  [fill  in  with  actual 
text  from  plan]? 

Facilitators 

What  kinds  of  things  help  or  would  have 
helped  to  implement  this  part  of  your 
plan  more  often  or  more  effectively? 

Barriers 

What  kind  of  things  have  or  could  have 
gotten  in  the  way  of  being  able  to 
implement  this  part  of  your  plan? 

Importance 

How  important  is  this  part  of  your  plan 
for  the  work  that  you  do? 

Sustainability 

Do  you  think  you'll  keep  doing  it?  Why 
or  why  not? 

Care  phase 

You  mentioned  in  your  Action  Plan  that  you  wanted  to  [fill 
in  with  actual  text  from  individual's  plan]  to  improve  care 
for  clients  with  co-occurring  disorders. 

Implementation  of 
Action  Plan 

How  regularly  do  you  [fill  in  with  actual 
text  from  plan]? 

Facilitators 

What  kinds  of  things  help  or  would  have 
helped  to  implement  this  part  of  your 
plan  more  often  or  more  effectively? 

Barriers 

What  kind  of  things  have  or  could  have 
gotten  in  the  way  of  being  able  to 
implement  this  part  of  your  plan? 

Importance 

How  important  is  this  part  of  your  plan 
for  the  work  that  you  do? 

Sustainability 

Do  you  think  you'll  keep  doing  it?  Why 
or  why  not? 

Relapse  prevention/ 
continuing  care  phase 

You  mentioned  in  your  Action  Plan  that  you  wanted  to 
[fill  in  with  actual  text  from  individual's  plan]  to  improve 
relapse  prevention/continuing  care  for  clients  with  co¬ 
occurring  disorders. 

Implementation  of 
Action  Plan 


How  regularly  do  you  [fill  in  with  actual 
text  from  plan]? 
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Table  D.1 — Continued 


Question  to 


Topic 

Initiate  Discussion 

1  Possible  Prompts/Probes 

Facilitators 

What  kinds  of  things  heip  or  wouid  have 
heiped  to  impiement  this  part  of  your 
plan  more  often  or  more  effectively? 

Barriers 

What  kind  of  things  have  or  could  have 
gotten  in  the  way  of  being  able  to 
implement  this  part  of  your  plan? 

Importance 

How  important  is  this  part  of  your  plan 
for  the  work  that  you  do? 

Sustainability 

Do  you  think  you'll  keep  doing  it?  Why 
or  why  not? 

**  THE  REMAINING  QUESTIONS/TOPICS  WERE  ASKED  WHEN  RELEVANT  BASED  ON 
THE  PARTICIPANT'S  SURVEY  RESPONSES. 

Administrator's 

Guidebook 

You  mentioned  in  your  survey  that  you  used  the 
Administrator's  Guidebook  to  [fill  in  what  they  said  in 
survey].  Can  you  tell  me  more  about  how  you  used  the 
Guidebook? 

Adoption  of 

Administrator's 

Guidebook 

Which  domains  did  you  work  on? 

Utility  of 

Administrator's 

Guidebook 

Tell  me  about  the  usefulness  of  the 
Guidebook  in  helping  you  expand  your 
program's  dual  diagnosis  capability. 

Sustainability  of 

Administrator's 

Guidebook 

Do  you  think  the  changes  will  last?  Why 
or  why  not? 

Barriers  to  using 

Administrator's 

Guidebook 

Tell  me  about  the  barriers  to  using  the 
Guidebook. 

Changes  made  using 

Administrator's 

Guidebook 

Talk  about  some  of  the  changes 
you  made  to  the  program  using  the 
Guidebook. 

Performance 

Measurement 

What  are  your  thoughts  about  using 
the  DDCAT  as  an  ongoing  measure  of 
program  performance? 

Screening  and 
Assessment  Curriculum 

Tell  me  about  your  use  of  the  Screening  and  Assessment 
Curriculum. 

(#1) 
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Table  D.1 — Continued 


Question  to 

Topic  Initiate  Discussion  Possible  Prompts/Probes 


Adoption  of 

Screening  and 

Assessment 

Curriculum 

Did  you  use  the  assessments  or  screeners 
in  the  curricuium?  Why  or  why  not? 

Utility  of  Assessment 

Curriculum 

Teii  me  about  the  usefuiness  of  the 
Screening  and  Assessment  Curricuium 
in  heiping  you  screen  and  assess  ciients 
with  COD. 

Sustainability 
of  Screening 
and  Assessment 

Curriculum 

Are  you  stiii  using  the  screening  and 
assessment  toois?  if  not,  why  not? 

Barriers  to  using 

Screening  and 

Assessment 

Curriculum 

Teii  me  about  the  barriers  to  using  the 
curricuium  to  heip  you  screen  and  assess 
ciients  with  COD. 

Integrating  Combined  Tell  me  about  your  use  of  the  Integrating  Combined 
Therapies  Curriculum  Therapies  Curriculum. 

(#2) 


Adoption  of 

Integrating 

Combined  Therapies 

Curriculum 

Did  you  use  the  iCT  curricuium  to  treat 
ciients  with  COD?  Why  or  why  not? 

Utility  of  ICT 

Curriculum 

Teii  me  about  the  usefuiness  of  the  iCT 
curricuium  in  heiping  you  treat  ciients 
with  COD. 

Sustainability  of 

Combined  Therapies 

Curriculum 

Are  you  stiii  using  the  iCT  curricuium?  if 
not,  why  not? 

Barriers  to  using 

Combined  Therapies 

Curriculum 

Teii  me  about  the  barriers  to  using  the 
curricuium  to  heip  you  treat  ciients  with 
COD. 

Cognitive-Behavioral  Tell  me  about  your  use  of  the  Cognitive  Behavioral  Therapy 
Therapy  Curriculum  (#3)  Curriculum. 


Adoption  of 

Cognitive-Behavioral 

Therapy  Curriculum 

Do  you  or  did  you  use  the  CBT 
curricuium  to  treat  ciients  with  COD?  Did 
you  use  the  materiais  provided?  Why  or 
why  not? 
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Table  D.1 — Continued 


Question  to 

Topic  Initiate  Discussion  Possible  Prompts/Probes 


Utility  of  CBT 
curriculum 

Tell  me  about  the  usefulness  of  the  CBT 
curriculum  In  helping  you  treat  clients 
with  COD. 

Sustainability  of 

Cognitive-Behavioral 

Therapy  Curriculum 

Are  you  still  using  the  CBT  curriculum?  If 
not,  why  not? 

Barriers  to  using 

Cognitive-Behavioral 

Therapy  Curriculum 

Tell  me  about  the  barriers  to  using  the 
curriculum  to  help  you  treat  clients  with 
COD. 

Medication  Tell  me  about  your  use  of  the  Medication  Management 

Management  Curriculum. 

Curriculum  (#4) 


Adoption  of 

Medication 

Management 

Curriculum 

Do  you  or  did  you  use  the  MM 
curriculum  to  assist  you  in  treating 
clients  with  COD?  Did  you  use  the 
materials  provided?  Why  or  why  not? 

Utility  of  MM 

Curriculum 

Tell  me  about  the  usefulness  of  the  MM 
curriculum  In  helping  you  treat  clients 
with  COD. 

Sustainability 
of  Medication 

Management 

Curriculum 

Are  you  still  using  the  MM  curriculum?  If 
not,  why  not? 

Barriers  to  using 

Medication 

Management 

Curriculum 

Tell  me  about  the  barriers  to  using  the 
curriculum  to  help  you  treat  clients  with 
COD. 
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In  recent  years,  the  prevalence  of  psychological  health  conditions  among 
military  service  members  has  increased  substantially.  Patients  with  co-occurring 
psychological  health  and  substance  abuse  conditions  have  unique  treatment 
needs,  and  evidence  suggests  that  integrated  treatment  for  these  conditions  may 
be  more  effective  than  treating  each  separately.  The  Navy's  Bureau  of  Medicine 
contracted  with  Hozelden  to  provide  training  for  clinicians  to  deliver  an  evidence- 
based  intervention  for  treating  patients  with  co-occurring  disorders.  The  Hozelden 
training,  the  Co-Occurring  Disorders  Clinician  Training  Program  (CODP),  began  in 
2008. 

RAND  evaluated  the  implementation  of  the  training  among  Substance  Abuse 
Rehabilitation  Program  (SARP)  personnel  to  understand  the  CODP  approach 
and  goals,  and  to  describe  the  training  program  and  materials  and  trainee 
perceptions,  identify  which  program  elements  were  implemented  and  sustained, 
identify  facilitators  and  barriers  to  implementation,  and  describe  programs' 
capabilities  to  provide  integrated  care. 

This  report  presents  the  results  and  makes  recommendations  for  improving 
training  of  SARP  personnel.  RAND  researchers  formulated  the  following 
recommendations:  Develop  a  training  plan,  obtain  leadership  support,  plan  for 
staff  turnover,  provide  consultation  after  training,  and  evaluate  training  efforts; 
ensure  that  clinician  training  focuses  on  materials  and  skills  most  relevant  to  their 
practice;  consider  requiring  that  all  service  members  receiving  care  from  a  SARP 
be  screened  for  substance  use  and  psychological  health  problems;  identify  and 
certify  select  sites  as  providing  enhanced  co-occurring  disorders  services;  and 
implement  measures  to  assess  the  quality  of  care  at  SARPs. 
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